Image enhancement method and system

ABSTRACT

An image enhancement method and system are provided. The image enhancement method and system are capable of improving image quality by performing gamma corrections on global and local illuminations and reflectance estimated from an input image, in consideration of dynamic range and contrast of the image, respectively. Particularly, in a case of a color image, the red, green, and blue (RGB) component images are converted into hue, saturation, and value (HSV) component images, and the global and local illuminations and reflectance estimated from the V component image. By converting the hue (H), saturation (S), and enhanced value (V) into RGB, an enhanced color image can be obtained.

PRIORITY

This application claims the benefit under 35 U.S.C. §119(a) of a Koreanpatent application filed in the Korean Intellectual Property Office onOct. 30, 2006, and assigned Serial No. 2006-0105912, the entiredisclosure of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image enhancement technique. Moreparticularly, the present invention relates to an image enhancementmethod and system that are capable of improving image quality byperforming gamma corrections on global and local illuminations andreflectance estimated from an input image, respectively.

2. Description of the Related Art

With the widespread popularity of digital cameras (including digitalcamera-equipped mobile phones), taking images has become as commonplaceas making a telephone call. However, most images, such as those obtainedwith a portable digital camera, are obtained in an irregularillumination environment such as fluorescent lamp, sunlight, and streetlamp, rather than in an artificially prepared regular illuminationenvironment, e.g. in a studio. Such an image is likely to bedeteriorated with an excessively shaded or bright region. Accordingly,various techniques have been proposed for improving the quality of thedeteriorated image.

Well known techniques for improving image quality include intensitytransformation, histogram modeling, and homomorphic filtering, retinexand Kimmel methods based on an image formation model.

The intensity transformation technique is disclosed by R. C. Gonzalezand R. E. Woods in “Digital Image Processing,” Reading, Mass.,Addison-Wesley, 1992 and W. K. Pratt, “Digital Image Processing,” 2nded. New York, Wiley, 1991.

The histogram modeling technique is disclosed by A. K. Jain in“Fundamentals of Digital Image Processing,” Enblewood Cliffs, N.J.,Prentice-Hall, 1989.

The homomorphic filtering method based on the image formation model isdisclosed by J. S. Lim in “Two-Dimensional Signal and Image Processing”Englewood Cliffs, N.J., Prentice-Hall, 1990.

The retinex method is disclosed by D. J. Jobson, Z. Rahman, and G. A.Woodell in “Properties and Performance of a Center/Surround Retinex,”IEEE Trans. Image Process., vol. 6, no. 3, pp. 451-462, March 1997, byM. Ogata, T. Tsuchiya, T. Kubozono, and K. Ueda in “Dynamic RangeCompression Based on Illumination Compensation,” IEEE Trans. ConsumerElectron., vol. 47, no. 3, pp. 548-558, August 2001, by R. Kimmel, M.Elad, D. Shaked, R. Keshet, and I. Sobel in “A Variational Framework forRetinex,” Int. J. Comput. Vis., vol. 52, no. 1, pp. 7-23, January 2003,and by D. J. Jobson, Z. Rahman, and G. A. Woodell in “A Multi-ScaleRetinex for Bridging the Gap Between Color Images and the HumanObservation of Scenes,” IEEE Trans. Image Process., vol. 6, no. 7, pp.965-976, July 1997.

The Kimmel method is disclosed in U.S. Pat. No. 6,941,028 and by R.Kimmel, M. Elad. D. Shaked, R. Keshet, and I. Sobel in “A VariationalFramework for Retinex,” Int. J. Comput. Vis., vol. 52, no. 1, pp. 7-23,January 2003.

All of these methods improve the qualities of images by decreasing adynamic range of an input image or increasing a contrast of the image.

The intensity transformation method, histogram modeling method,homomorphic filtering method, retinex method and Kimmel method will eachbe briefly described below. The intensity transformation method improvesthe quality of an input image by transforming brightness values of theimage using a linear function, log function or power function (R. C.Gonzalez and R. E. Woods, “Digital Image Processing,” Reading, Mass.,Addison-Wesley, 1992 and W. K. Pratt, “Digital Image Processing,” 2nded. New York, Wiley, 1991). In the case of using a linear function asthe transformation function, it is called contrast stretching orgain/offset correction method. In the case of using a power function, itis called gamma correction method. These intensity transformationmethods are known to be easily implemented. However, since only onetransformation function is used for an entire image, it is difficult,when there exist several regions in the image of which contrasts aredifferent from each other, to increase the contrasts at the same time.

The histogram modeling method obtains an improved image by transformingthe input image into an image which has a required form of histogram. Inthe case that the required form of histogram is uniform, it is called ahistogram equalization method. Although the histogram equalizationmethod can be easily implemented as a brightness transformation, it isdifficult to increase the contrast of different areas in an image at thesame time. For example, the contrast of a simple background distributedin a small area decreases, while the contrast of a simple backgrounddistributed in a large area increases. Accordingly, the processed imageis unnaturally expressed rather than improved.

In order to overcome this shortcoming, a modified histogram equalizationmethod has been proposed in which an input image is divided into aplurality of blocks and the histogram equalization is applied for eachblock. Such an algorithm is disclosed by R. C. Gonzalez and R. E. Woodsin “Digital Image Processing,” Reading, Mass., Addison-Wesley, 1992; byS. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T.Greer, B. T. H. Romeny, J. B. Zimmerman, and K. Zuiderveld in “AdaptiveHistogram Equalization and its Variations,” Comput. Vis., Graph., ImageProcess., vol. 39, no. 3, pp. 355-368, September 1987 and by J. Y. Kim,L. S. Kim; and S. H. Hwang in “An Advanced Contrast Enhancement UsingPartially Overlapped Sub-block Histogram Equalization,” IEEE Trans.Circuits Syst. Video Technol., vol. 11, no. 4, pp. 475-484, April 2001.

The R. C. Gonzalez and R. E. Woods, “Digital Image Processing” Reading,Mass., Addison-Wesley, 1992, discloses a method in which the input imageis divided into blocks of pixels; the S. M. Pizer, E. P. Amburn, J. D.Austin, R. Cromartie, A. Geselowitz, T. Greer, B. T. H. Romeny, J. B.Zimmerman, and K. Zuiderveld, “Adaptive Histogram Equalization and itsVariations,” Comput. Vis., Graph., Image Process., vol. 39, no. 3, pp.355-368, September 1987, discloses a method in which the blocks aredivided without overlapping with each other; the J. Y. Kim, L. S. Kim,and S. H. Hwang, “An Advanced Contrast Enhancement Using PartiallyOverlapped Sub-block Histogram Equalization,” IEEE Trans. Circuits Syst.Video Technol., vol. 11, no. 4, pp. 475-484, April 2001 discloses ahistogram equalization method with partially overlapped blocks; and theV. Buzuloiu, M. Ciuc, R. M. Rangayyan, and C. Vertan,“Adaptive-neighborhood Histogram Equalization of Color Images,” J.Electron. Imag., vol. 10, no. 2, pp. 445-459, April 2001 discloses ahistogram equalization method in which the image is divided into blocksin consideration of the variations of the size and form of the inputimage. Also, the V. Buzuloiu, M. Ciuc, R. M. Rangayyan, and C. Vertan,“Adaptive-neighborhood Histogram Equalization of Color Images,” J.Electron. Imag., vol. 10, no. 2, pp. 445-459, April 2001 discloses ageneralized histogram equalization method.

The homomorphic filtering method applies a logarithmic function to theinput image and then applies a linear low-pass filter (LPF) and a linearhigh-pass filter (HPF) to the log signal of the input image. From therespective output, the log signal of the illumination and the log signalof the reflectance are estimated. The estimated log signal of theillumination is multiplied by a value less than 1 for reducing thedynamic range and the estimated log signal of reflectance is multipliedby a value greater than 1 for increasing the contrast. Finally, the twosignals are summed at an adder and an exponential function is applied tothe sum so as to return to the spatial domain from the log domain. Inthis method, by multiplying a value greater than 1 to the log signal ofthe reflectance, the contrast of a bright region in the output image mayincrease in relation to the contrast of a dark region. So, thecharacteristic of the human visual system (HVS) is not well reflected.The nonlinear log operation generates harmonics in the frequencies ofthe input image. This results in variation of the frequency spectrum soas to cause unreliability of estimation on the log signals of theillumination and the reflectance.

Meanwhile, the retinex method based on the retinex theory proposed by E.Land in “An Alternative Technique for the Computation of the Designatorin the Retinex Theory of Color Vision,” Proc. Nat. Acad. Sci., vol. 83,pp. 3078-3080, May 1986, assumes that the HVS recognizes colors withoutaffection of the illumination.

The retinex method applies a linear LPF for estimating the illuminationfrom the input image and then the reflectance is estimated by removingthe estimated illumination from the input image. Only the reflectance isnext emphasized. The retinex method proposed by D. J. Jobson, Z. Rahman,and G. A. Woodell in “Properties and Performance of a Center/surroundRetinex,” IEEE Trans. Image Process., vol. 6, no. 3, pp. 451-462, March1997 estimates the illumination using a Gaussian-form linear LPF andsubtracts the log signal of the estimated illumination from the logsignal of the input image so as to estimate the log signal of thereflectance. Without returning to the spatial domain, the reflectancesignal estimated in the log domain is used as the output image. Finally,the brightness range of the output image is adjusted by a gain/offsetcorrection in accordance with that of the output device. In the methodproposed by Jobson et al., the reflectance of the output image isestimated with a difference between the input image and an illuminationestimated using the linear LPF.

Since the output image is given as the estimated reflectance, the outputimage is characterized in that a dark region becomes darker and a brightregion becomes brighter along the edge at which the brightness tends tovary abruptly such that a halo effect appears. However, in a case thatthe support region of the linear LPF is narrow, the estimatedillumination is smoothed in a small area around the edge. So the haloeffect appears in a small area around an edge and the local contrastincreases. On the other hand, if the support region of the linear LPF iswide, the estimated illumination is smoothed in large area. At thistime, the halo effect appears in a large area and the global contrastincreases. In order to increase the local contrast and the globalcontrast simultaneously, a method using the weighted sum of outputimages by using three linear LPFs having different support regions hasbeen proposed. This method is disclosed by D. J. Jobson, Z. Rahman, andG. A. Woodell in “A Multi-scale Retinex for Bridging the Gap BetweenColor Images and the Human Observation of Scenes,” IEEE Trans. ImageProcess., vol. 6, no. 7, pp. 965-976, July 1997. Such a retinex methodis called multi-scale retinex (MSR), whereas one that uses only onelinear low-pass filter is called single-scale retinex (SSR).

The method proposed by Kimmel is disclosed by R. Kimmel, M. Elad, D.Shaked, R. Keshet, and I. Sobel in “A Variational Framework forRetinex,” Int. J. Comput. Vis., vol. 52, no. 1, pp. 7-23, January 2003.In this method, the illumination is iteratively estimated usingquadratic programming (QP) under a constraint that the illumination ofthe output image is brighter than or equal to that of the input imageover the entire region. The reflectance is estimated by inversing theestimated illumination and multiplying it with the input image. In orderto compress the dynamic range of the estimated illumination, the gammacorrection is applied and then it is multiplied with the estimatedreflectance so as to obtain an output image.

The homomorphic filtering method, retinex (SSR, MSR), and Kimmel methodwill be described hereinafter with reference to FIGS. 1 to 8 in moredetail.

In the conventional image formation model, a grayscale image or acomponent image of a color image f(x,y) can be expressed as amultiplication of illumination l(x,y) and reflectance r(x,y) in equation(1):

f(x,y)=l(x,y)·r(x,y)   (1)

where it is assumed that the illumination l(x,y) is slowly changed andits frequency spectrum is mostly distributed at low frequency band.Also, it is assumed that the reflectance r(x,y) is rapidly changed dueto the reflection characteristics on the surface of an object and itsfrequency spectrum is mostly distributed at high frequency band. In theconventional image enhancement technique based on the image creationmodel of equation 1, at least one of the illumination and reflectance isestimated. The estimated illumination is then processed so as to reduceits dynamic range and the estimated reflectance is processed to increaseits contrast. An output image is obtained by combining these processedcomponents. Next, the output image is used as it is or is corrected bygain/offset correction for matching the brightness value range of anoutput device.

Homomorphic Filtering Method

As shown in FIG. 1, the log signal log f(x,y) of the input image isobtained by applying the logarithmic function at logarithmic functionblock 101. Based on the image formation model of equation 1, the logf(x,y) is expressed as the sum of the log signal log l(x,y) of theillumination and the log signal log r(x,y) of the reflectance as inequation (2).

log f(x,y)=log l(x,y)+log r(x,y)   (2)

The log signal log l(x,y) of the illumination of equation (2) isestimated by applying the LPF 103 to the log signal log f(x,y) of theinput image under the assumption that the frequency spectrum isdistributed at low frequency band as in equation (3). The log signal logr(x,y) of the reflectance is estimated by applying the HPF 105 as inequation (4) under the assumption that the frequency spectrum of the logsignal is of the reflectance is distributed at high frequency band,

log {circumflex over (l)}(x,y)=LPF[log f(x,y)]  (3)

log {circumflex over (r)}(x,y)=HPF[log f(x,y)]  (4)

where log {circumflex over (l)}(x,y) is the estimated log signal ofillumination, log {circumflex over (r)}(x,y) is the estimated log signalof reflectance. The estimated log signal log {circumflex over (l)}(x,y)of the illumination is multiplied by a constant a less than 1 at themultiplier 107 for decreasing the dynamic range, and the estimated logsignal log {circumflex over (r)}(x,y) of the reflectance is multipliedby a constant β greater than 1 at the multiplier 108 for increasing thecontrast. The output signals of the multipliers 107 and 108 are summedat an adder 109 and then the output signal of an adder 109 is processedby the exponential function block 111 so as to be reconstructed. At thistime, getting back to the spatial domain obtains the output image{circumflex over (f)}(x,y). From the comparison of an input imageillustrated in FIG. 2A and an output image illustrated in FIG. 2B,homomorphic filtering method improves the quality of the image. However,the homomorphic filtering method requires the nonlinear log operationsuch that the log signal f(x,y) of the input image generates harmonicsof the frequency spectrum. The harmonics of the frequency spectrum makeit difficult to estimate the log signals of the illumination andreflectance. Particularly, the output image {circumflex over (f)}(x,y)obtained using the homomorphic filtering method can be expressed byequation (5).

{circumflex over (f)}(x,y)=exp(α·log {circumflex over (l)}(x,y)+β·log{circumflex over (r)}(x,y))   (5)

Rearranging the right terms of the equation 5 with respect to theestimated illumination {circumflex over (l)}(x,y) and the estimatedreflectance {circumflex over (r)}(x,y) the output image {circumflex over(f)}(x,y) can be expressed in a form of the multiplication of the twoestimated components enhanced by the gamma correction as shown inequation 6:

{circumflex over (f)}(x,y)=({circumflex over (l)}(x,y))^(α)·({circumflexover (r)}(x,y))^(β)  (6)

where α<1 and β>1. In equation (6), a gamma correction is applied with agamma factor β greater than 1 to the reflectance in the range of 0˜1,the contrast of the image increases in the bright region relative to thedark region, as shown in FIG. 2B. This is because the homomorphicfiltering method does not consider that the HVS is more sensitive to thecontrast of the dark region than that of the bright region.

FIG. 2A illustrates an input image having strong edges at the centerpart and dark and weak edges at the left and right parts. FIG. 2Billustrates the output image obtained by applying the homomorphicfiltering to the input image of FIG. 2A. In this case, the log signal ofthe illumination is estimated by separately applying a 1-dimensionalGaussian LPF with a window of 1×5 size in horizontal and verticaldirections to the log signal of the input image. The log signal of thereflectance is estimated by subtracting the log {circumflex over(l)}(x,y) from the log f(x,y). The constants α and β are 0.5 and 1.5,respectively. From the output image of FIG. 2B, it is shown that thebright region around the tower shows strong contrast relative to thedark region around trees.

Retinex Method

Typically, retinex methods are classified into SSR and MSR.

FIG. 3 is a block diagram illustrating the SSR method proposed by D. J.Jobson, Z. Rahman, and G. A. Woodell in “Properties and Performance of aCenter/surround Retinex,” IEEE Trans. Image Process., vol. 6, no. 3, pp.451-462, March 1997.

In the SSR, the illumination is estimated by applying a linear LPF 301to the input image f(x,y). The input image is also directed to logfunction block 305. The output image {circumflex over (f)}(x,y) isobtained at adder 307 from a signal taken by removing a log signal log{circumflex over (l)}(x,y) obtained from log function block 303 of theestimated illumination, from the log of the input image obtained fromlog function block 305 as shown in equation (9):

{circumflex over (f)}(x,y)=log f(x,y)−log {circumflex over (l)}(x,y)  (9)

where {circumflex over (l)}(x,y) is the estimated illumination through alinear low-pass filtering by equation 10.

{circumflex over (l)}(x,y)=LPF[f(x,y)]  (10)

The output image {circumflex over (f)}(x,y) of equation 9 can beexpressed with respect to the estimated illumination {circumflex over(l)}(x,y) by equation (11):

{circumflex over (f)}(x,y)=({circumflex over (l)}(x,y))⁰·log({circumflexover (r)}(x,y))   (11)

where {circumflex over (r)}(x,y) is the reflectance estimated with

${\hat{r}\left( {x,y} \right)} = {\frac{f\left( {x,y} \right)}{\hat{l}\left( {x,y} \right)}.}$

The SSR represented by equation (11) applies a gamma collection with afactor of α=0 to the estimated illumination {circumflex over (l)}(x,y)such that the affection of illumination is excluded unlike in thehomomorphic filtering method. However, the logarithmic function, whosetransfer characteristic is similar to the gamma correction with a factorof β=0.3, is applied to the reflectance {circumflex over (r)}(x,y) suchthat the contrast increases excessively at a dark region.

Finally, in order to obtain final output image {tilde over (f)}(x,y)whose brightness value range is adjusted to that of the output device, again/offset correction is applied to the output image {circumflex over(f)}(x,y) by gain/offset correction function block 309 as in equation12:

{tilde over (f)}(x,y)=a[{circumflex over (f)}(x,y)+b]  (12)

where a and b are constants used for the gain/offset correction. Thevalues of a and b affect the final output image {tilde over (f)}(x,y).As disclosed by D. J. Jobson, Z. Rahman and G. A. Woodell in “Propertiesand Performance of a Center/surround Retinex,” IEEE Trans. ImageProcess., vol. 6, no. 3, pp. 451-462, March 1997, the same values of aand b are used for all images.

FIG. 4 is a diagram illustrating a 1-dimensional signal presenting anedge component of an input image and estimated illumination andreflectance in the SSR method of FIG. 3. In FIG. 4, a linear LPFsmoothes the edge component of the input image 401 to estimate theillumination 402 such that the reflectance 403 estimated by areflectance estimator 405 abruptly decreases or increases around theedge. This causes a halo effect in which the dark region of the inputimage becomes darker and the bright region of the input image becomesbrighter in the output image. The edge region of the smoothed image inthe illumination estimated by the linear low-pass filtering varies insize and smoothing degree according to the support region of the filter.The edge region to be smoothed becomes narrow or wide according to thewidth of the support region of the filter. At this time, the smoothingdegree also increases or decrease and the scale of the halo effectincreases or decreases. If the support region of the filter is narrow,the halo effect occurs in a small area so the local illumination can bewell estimated, resulting in an increase of the local contrast of theimage. Conversely, if the support region of the filter is wide, the haloeffect occurs in a large area so the global illumination can be wellestimated, resulting in an increase of the global contrast of the image.Since the halo effect and the contrast vary according to the supportregion of the filter, it is required to adjust the support region of thefilter. FIGS. 5A to 5C illustrate SSR output images to the input imageof FIG. 2A according to the width of the support region of the linearLPF. The constants a and b for the gain/offset correction in equation 12have values of 192 and −30, respectively, as in D. J. Jobson, Z. Rahman,and G. A. Woodell, “A Multi-scale Retinex for Bridging the Gap BetweenColor Images and the Human Observation of Scenes,” IEEE Trans. ImageProcess. vol. 6, no. 7, pp. 965-976, July 1997. If the support region ofthe filter is set to 15 pixels, the halo effect in the output image ofSSR in FIG. 5A appears at small areas around the roof and the tower. Itshows that the local contrast of the image increases but the globalcontrast does not increase well. FIG. 5C illustrates an output image ofSSR when the support region of the filter is set to 500 pixels. In thiscase, the halo effect appears in wide areas around the roof and thetower. This means that the global contrast increases but the localcontrast does not increase. FIG. 5B illustrates an output image of SSRwhen the support region of the filter is set to 150 pixels. In thiscase, the halo effect appears to an extent between those of FIGS. 5A and5C. The MSR is proposed by D. J. Jobson, Z. Rahman, and G. A. Woodell in“A Multi-scale Retinex for Bridging the Gap Between Color Images and theHuman Observation of Scenes,” IEEE Trans. Image Process., vol. 6, no. 7,pp. 965-976, July 1997. The MSR is a technique for compensating thevariational characteristics of the SSR output images according to thesupport region of the linear LPF for estimating the illumination. In theMSR, an output image {circumflex over (f)}_(MSR)(x,y) is obtained by aweighted sum of the SSR output images taken by using the linear LPFshaving different support regions as expressed in equation 13:

$\begin{matrix}{{{\hat{f}}_{MSR}\left( {x,y} \right)} = {\sum\limits_{n = 1}^{N}{w_{n}{{\hat{f}}_{n}\left( {x,y} \right)}}}} & (13)\end{matrix}$

where N is a number of filters, {circumflex over (f)}_(n)(x,y) is an SSRoutput image taken through the nth filter (n=1,2, . . . ,N), and w_(n)is a weighting factor. The brightness range of the MSR output image{circumflex over (f)}_(MSR)(x,y) is adjusted to that of the outputdevice by applying the gain/offset correction. FIG. 6 illustrates an MSRoutput image obtained from a weighted sum of the SSR output images ofthe input image of FIG. 2A with the LPFs of which support regions are15, 150, and 500 pixels, respectively. Here, the constants a and b forthe gain/offset correction in equation (12) are set to 192 and −30 as inequation 8. As shown in FIG. 6, since the advantageous factors of theSSR output images taken through the filters having different supportregions are partially combined, both the global and local contrasts arewell increased. Although the halo effect is attenuated to some extent,the MSR output image has a residual halo effect because the most SSRoutput images have halo effects.

FIG. 7 is a block diagram illustrating the Kimmel method. The Kimmelmethod is disclosed in the U.S. Pat. No. 6,941,028 and by R. Kimmel, M.Elad, D. Shaked, R. Keshet, and I. Sobel in “A Variational Framework forRetinex,” Int. J. Comput. Vis., vol. 52, no. 1, pp. 7-23, January 2003.In this method, the illumination is iteratively estimated using a QP 701under the constraints that the illumination varies slowly in all regionsincluding boundaries and its brightness is greater than or equal to theinput image etc. After the illumination is estimated, the reflectance isestimated by passing the estimated illumination {circumflex over(l)}(x,y) estimated according to equation (8) through the inverter 707and multiplying the output of the inverter 707 with the input imagef(x,y) at the multiplier 709. The estimated illumination {circumflexover (l)}(x,y) is gamma-corrected at the corrector 703 for decreasingthe dynamic range. Finally, the output image is obtained by multiplyingthe gamma-corrected illumination with the estimated reflectance at themultiplier 705 as following equation 14:

{circumflex over (f)}(x,y)={circumflex over (l)}(x,y)^(α) ·{circumflexover (r)}(x,y)   (14)

where {circumflex over (r)}(x,y) is an estimated reflectance. Thismethod is advantageous in estimating the reflectance satisfying theconstraints. However, the iterative estimation of illumination causesmassive computation and no enhancement operation is applied to the{circumflex over (r)}(x,y).

In the color image enhancement methods based on the conventional imageformation model, the image enhancement in the RGB color space can beexpressed by equation (15) by applying the image of equation (1) to therespective component images f_(i)(x,y),iε{R,G,B}.

f _(i)(x,y)=l _(i)(x,y)·r _(i)(x,y),iε{R,G,B}  (15)

On the basis of equation (15), the image enhancement can be applied toeach component image of the RGB color image. In this case, the red (R),green (G), and blue (B) components vary independently such that thecolor variations may be caused by variations of the ratio of the R, G,and B components. The color image enhancement methods are proposed inSSR (D. J. Jobson, Z. Rahman, and G. A. Woodell, “Properties andPerformance of a Center/surround Retinex,” IEEE Trans. Image Process.,vol. 6, no. 3, pp. 451-462, March 1997), MSR, and MSR with ColorRestoration (MSRCR) (D. J. Jobson, Z. Rahman, and G. A. Woodell, “AMulti-scale Retinex for Bridging the Gap Between Color Images and theHuman Observation of Scenes,” IEEE Trans. Image Process., vol. 6, no. 7,pp. 965-976, July 1997). These methods enable obtaining output colorimages {circumflex over (f)}_(i)(x,y),iε{R,G,B} only with the enhancedreflectance of equation (16).

{circumflex over (f)} _(i)(x,y)=({circumflex over (l)}_(i)(x,y))⁰·log({circumflex over (r)} _(i)(x,y)),iε{R,G,B}  (16)

In equation (16), it is noted that the output image {circumflex over(f)}_(i)(x,y),iε{R,G,B} is obtained regardless of the estimatedillumination of the input image.

In a case that the estimated reflectance of the R, G, and B componentimages have similar values, the R, G, B component ratio becomes 1:1:1such that a gray-world violation occurs, in which the color of theoutput image shifts to gray.

In order to overcome this problem, MSRCR is proposed. In the MSRCR, acolor restoration process is added as following equation (17):

{circumflex over (f)} _(MSRCR) _(i) (x,y)=C _(i)(x,y)·{circumflex over(f)} _(MSR) _(i) (x,y),iε{R,G,B}  (17)

where {circumflex over (f)}MSRCR _(i) (x,y) and {circumflex over(f)}_(MSR) _(i) (x,y) are MSRCR output image and an MSR output image ofthe component image {circumflex over (f)}_(i)(x,y),iε{R,G,B}, andC_(i)(x,y) is a weighting function expressed as equation 18 for colorrestoration:

$\begin{matrix}{{{C_{i}\left( {x,y} \right)} = {c \cdot {\log\left\lbrack {d \cdot \frac{f_{i}\left( {x,y} \right)}{\sum\limits_{j = 1}^{S}{f_{j}\left( {x,y} \right)}}} \right\rbrack}}},{ \in \left\{ {R,G,B} \right\}}} & (18)\end{matrix}$

where c and d are constants for color restoration, S is the number ofthe color components of the image. In equations 17 and 18, the RGBcomponent images of the MSRCR output color image are weighted accordingto the RGB component ratio of the input color image. Accordingly, theoutput image is presented with a color similar to that of the inputcolor image. Finally, the gain/offset correction of equation 12 isapplied to the MSRCR output image {circumflex over(f)}_(i)(x,y),iε{R,G,B} to fit the brightness value range of the outputimage to that of the output device. FIG. 8A illustrates an input colorimage, and FIGS. 8B and 8C illustrate MSR and MSRCR output color imagesfor the input color image of FIG. 8A. The constants a and b for thegain/offset correction of equation 12 are set to 192 and −30,respectively, and the constants c and d for the color restoration ofequation 18 are set to 46 and 125, respectively, for the MSRCR asproposed by D. J. Jobson, Z. Rahman, and G. A. Woodell, in “AMulti-scale Retinex for Bridging the Gap Between Color Images and theHuman Observation of Scenes,” IEEE Trans. Image Process., vol. 6, no. 7,pp. 965-976, July 1997. In the MSR output color image of FIG. 8B, it isshown that the contrast of the dark area in the input color image iswell enhanced, but the gray-world violation appears. Although thegray-world violation is attenuated to an extent in the MSRCR outputcolor image of FIG. 8C, residual gray-world violation appears. That is,the MSRCR cannot be a solution for avoiding the gray-world violationeffect occurred in the MSR.

SUMMARY OF THE INVENTION

An aspect of the present invention is to address at least the abovementioned problems and to provide at least the advantages describedbelow. Accordingly, it is an aspect of the present invention to providean image enhancement method and system, particularly, for an imagedegraded by irregular illumination.

It is another aspect of the present invention to provide an imageenhancement method and system that are capable of restraining the haloeffect by applying a gamma correction with different factors to globalillumination, local illumination and reflectance estimated from an inputimage.

It is another aspect of the present invention to provide an imageenhancement method and system that are capable of improving the qualityof an image by enhancing global and local illuminations and reflectanceestimated from a value (V) component image of the input color imageunder the assumption of white-light illumination.

It is another aspect of the present invention to provide an imageenhancement method and system that are capable of effectively improvingan image quality by restraining the halo effect with estimation ofglobal and local illuminations and reflectance of an image.

In accordance with an aspect of the present invention, the above andother objects are accomplished by an image enhancement method. The imageenhancement method includes estimating the illumination from an inputimage, estimating the reflectance by removing the estimated illuminationfrom the input image, correcting the dynamic range of the estimatedillumination, correcting the contrast using the estimated reflectanceand adjusting a brightness value range of an output image obtained withthe corrected dynamic range and contrast to match a brightness valuerange of an output device.

In accordance with another aspect of the present invention, the aboveand other objects are accomplished by a color image enhancement method.The color image enhancement method includes converting RGB componentimages of an input image into HSV component images, estimating a globalillumination from V component image among HSV component images,estimating the local illumination using the estimated globalillumination and the V component image, estimating a reflectance usingthe estimated local illumination and the V component image, enhancingthe global and local illuminations and the reflectance for decreasingthe dynamic ranges of the estimated global and local illuminations andincreasing the contrast of the estimated reflectance, outputting anenhanced V by matching the brightness range of an intermediate imageobtained by multiplying the global and local illuminations and thereflectance to that of an output device and reproducing an RGB image bycombining the enhanced brightness component image with the hue (H) andsaturation (S) component images while converting the RGB componentimages into HSV component images.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainexemplary embodiments of the present invention will be more apparentfrom the following detailed description in conjunction with theaccompanying drawings, in which:

FIG. 1 is a block diagram illustrating a conventional image enhancementsystem using a homomorphic filtering process;

FIGS. 2A and 2B illustrate images before and after applying ahomomorphic filtering process of FIG. 1;

FIG. 3 is a block diagram illustrating a conventional image enhancementsystem using a SSR method;

FIG. 4 is a diagram illustrating a 1-dimensional signal presenting edgecomponent of an input image and estimated illumination and reflectancein SSR method of FIG. 3.

FIGS. 5A to 5C illustrate output images through the SSR method when aneffect region of a linear LPF is set to 15, 150, and 500 pixels,respectively;

FIG. 6 illustrates an output image of a conventional image enhancementmethod using MSR when the support regions of the LPFs are 15, 150, and500 pixels, respectively;

FIG. 7 is a block diagram illustrating a conventional image enhancementsystem using Kimmel method;

FIGS. 8A to 8C illustrate an input image and output images generated byapplying MSR and MSRCR methods, respectively;

FIG. 9 is a block diagram illustrating a configuration of an imageenhancement system according to an exemplary embodiment of the presentinvention;

FIG. 10 is a block diagram illustrating a configuration of a black andwhite image enhancement system according to an exemplary embodiment ofthe present invention;

FIGS. 11A and 11B are flowcharts illustrating a black and white imageenhancement method according to an exemplary embodiment of the presentinvention.

FIGS. 12A and 12B illustrate an input image and global illuminationestimated through the procedure of FIGS. 11A and 11B;

FIGS. 13A and 13B illustrate local illumination and reflectanceestimated from the input image of FIG. 12A;

FIG. 14 is a block diagram illustrating a configuration of a color imageenhancement system according to an exemplary embodiment of the presentinvention;

FIG. 15 is a block diagram illustrating a detailed configuration of acolor image enhancement system of FIG. 14;

FIGS. 16A and 16B are flowcharts illustrating a color image enhancementmethod according to an exemplary embodiment of the present invention;

FIGS. 17A and 17B illustrate images obtained by the color imageenhancement method of FIGS. 16A and 16B with and without adopting JND;

FIGS. 18A to 18C illustrate partially enlarged input image and outputimages obtained by a color image enhancement method of FIGS. 16A and 16Bwith and without adopting JND;

FIGS. 19A to 19D illustrate a NASA1 input color image, output colorimages by a histogram equalization method, MSRCR method, and an imageenhancement method according to an exemplary embodiment of the presentinvention;

FIGS. 20A to 20D illustrate a NASA2 input color image, output colorimages by a histogram equalization method, MSRCR method, and imageenhancement method according to an exemplary embodiment of the presentinvention; and

FIGS. 21A to 21D illustrate a CCD1 input color image, output colorimages by a histogram equalization method, MSRCR method, and an imageenhancement method according to an exemplary embodiment of the presentinvention.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features andstructures.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of exemplaryembodiments of the present invention as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the embodiments described hereincan be made without departing from the scope and spirit of theinvention. Also, detailed descriptions of well-known functions,structures and constructions incorporated herein may be omitted forclarity and conciseness.

Certain terms are used in the following description for convenience andreference only and are not to be construed as limiting. In the followingdetailed descriptions, only the exemplary embodiments of the inventionhave been shown and described, simply by way of illustration of the bestmode contemplated by the inventors of carrying out the invention. Aswill be realized, the invention is capable of modification in variousobvious respects, all without departing from the invention. Accordingly,the drawings and descriptions are to be regarded as illustrative innature and not restrictive.

The following definitions are provided to enable a clear and consistentunderstanding of the detailed description and the claims. Unlessotherwise noted, terms are to be understood according conventional usageby those skilled in the relevant art.

An “illumination or lighting” is a degree of visibility of environmentby which to see an object and expressed by l, and a “reflectance” is thefraction of incident radiation reflected by a surface and is expressedby r. A “global illumination” is the illumination over the entire imageincluding an object estimated from a predetermined size of image takenby a camera and is expressed by l_(G), and a “local illumination” is theillumination at a specific area of the entire image and is expressed byl_(L).

An “estimation” is a state to be predicted before determining the globaland local illuminations and the reflectance.

A “gamma correction” is a process for improving the global and localilluminations and reflectance by decreasing the dynamic range andincreasing the contrast.

FIG. 9 is a block diagram illustrating an image enhancement systemaccording to an exemplary embodiment of the present invention.

In FIG. 9, an input image is enhanced in the illumination and thereflectance. The image enhancement system includes a demodulator 901 forestimating the illumination l(x,y) by applying an envelope detector toan input image f(x,y) an inverter 903 for inverting the illuminationoutput from the demodulator 901, a first multiplier 905 for multiplyingthe estimated illumination {circumflex over (l)}(x,y) with the inputimage f(x,y) for estimating a reflectance r(x,y) a first corrector 909for decreasing the dynamic range of the estimated illumination{circumflex over (l)}(x,y) output from the demodulator 901, a secondcorrector 907 for correcting the estimated reflectance {circumflex over(r)}(x,y) so as to increase the contrast, a second multiplier 911 formultiplying the outputs from the first and second correctors 909 and907, and a third corrector 913 for correcting the estimated image{circumflex over (f)}(x,y) output from the second multiplier 911 so asto adjust the brightness of the output image {circumflex over (f)}(x,y)to that of an output device.

The first to third correctors 909, 907, and 913 perform correctionsusing a gamma or log function.

The demodulator 901 estimates the illumination l(x,y) from the inputimage f(x,y) in the following manner. Assuming that a bias is added toan illumination l(x,y) in the image formation model of equation 1, theinput image f(x,y) can be expressed with an amplitude modulation (AM) ofthe illumination l(x,y) with the reflectance r(x,y). Accordingly, theillumination l(x,y) can be estimated using an envelope detector, whichis a kind of AM demodulator, as following equation (19):

{circumflex over (l)}(x,y)=Env[f(x,y)]  (19)

where Env[] is the envelope detector.

In the homomorphic filtering method, the envelope detector 901 iscomposed by cascading log(), linear LPF, and exp(). In the SSR and MSRproposed by Jobson et al., a Gaussian-form linear LPF is used. In theKimmel method, the QP 701 (see FIG. 7) is used. The reflectance r(x,y)estimation can be generalized by dividing the input image f(x,y) withthe estimated illumination {circumflex over (l)}(x,y). In order toestimate the reflectance r(x,y) the estimated illumination {circumflexover (l)}(x,y) is inverted by the inverter 903 and then the output ofthe inverter 903 is multiplied with the input image f(x,y) at the firstmultiplier 905. The estimated reflectance {circumflex over (r)}(x,y) andillumination {circumflex over (l)}(x,y) are gamma-corrected at therespective first and second correctors 909 and 907 as in equation 20,and the outputs of the first and second correctors 909 and 907 aremultiplied at the second multiplier 911 so as to be output as anintermediate image. The intermediate image is then processed by thethird corrector 913 such that an output image {tilde over (f)}(x,y)whose brightness range is adjusted to that of the output device isgenerated.

{circumflex over (f)}(x,y)=g ₁({circumflex over (l)}(x,y))·g₂({circumflex over (r)}(x,y))   (20)

where g₁() is a function for decreasing the dynamic range of theestimated illumination {circumflex over (l)}(x,y) output from thedemodulator 901, and g₂() is a function for increasing the contrast ofthe estimated reflectance {circumflex over (r)}(x,y) output from thefirst multiplier 905. As the g₁() and g₂() for the first and secondcorrectors 909 and 907, the power function or log function can be used.In the homomorphic filtering method, g₁() and g₂() are expressed inthe form of ()^(α) and ()^(β) (α<1, β>1). The SSR and MSR proposed byJobson et al. use the power function ()⁰ and the log function log(),and the Kimmel method uses the power functions such as ()^(α) and()^(β) (α<1, β=1). The outputs of the first and second correctors 909and 907 are multiplied by the second multiplier 911, and the output ofthe second multiplier 911 is input to the third corrector 913 so as tooutput {circumflex over (f)}(x,y). The third corrector 913 processes theoutput of the second multiplier 911 for adjusting the brightness rangeof the output image to that of the output device. The output image{tilde over (f)}(x,y) is obtained by equation 21:

{tilde over (f)}(x,y)=g ₃({circumflex over (f)}(x,y))   (21)

where g₃() is a function for adjusting the brightness range of theoutput image to that of the output device. The function g₃() is notused in the homomorphic filtering methods and Kimmel method, and g₃()in the SSR and MSR proposed by Jobson et al., is used with gain/offsetcorrection.

FIG. 10 is a block diagram illustrating an image enhancement systemaccording to another exemplary embodiment of the present invention. Inthis embodiment, a black and white image enhancement system is proposed.

Referring to FIG. 10, the image enhancement system includes a globalillumination estimator 1004, a local illumination estimator 1006, areflectance estimator 1008, an illumination/reflectance enhancer 1010,and an image formatting and enhanced component output unit 1016.

The global illumination estimator 1004 is implemented with the firstenvelope detector 111 for estimating the global illumination l_(G)(x,y)by envelope detection through an AM demodulation on the input image.

The local illumination estimator 1006 includes the first inverter 113for inverting the estimated global illumination {circumflex over(l)}_(G)(x,y) output from the first envelope detector 111, the firstmultiplier 115 for multiplying the output of the first inverter 113 andthe input image f(x,y), and the second envelope detector 116 forestimating the local illumination l_(L)(x,y) by envelope detectionthrough an AM modulation on the output of the first multiplier 115.

The reflectance estimator 1008 includes the second inverter 118 forinverting the output of the second envelope detector 116 and a secondmultiplier 120 for estimating the reflectance r(x,y) by multiplying theoutput of the second inverter 118 and the output of the first multiplier115.

The illumination/reflectance enhancer 1010 includes the first and secondcorrectors 119 and 117 for decreasing dynamic ranges of the global andlocal illuminations, respectively, estimated by the global illuminationestimator 1004 and the local illumination estimator 1006, and a thirdcorrector 121 for increasing the reflectance estimated by thereflectance estimator 1008.

The image format and enhanced component output unit 1016 includes athird multiplier 122 for multiplying the outputs of the first and secondcorrectors 119 and 117 of the illumination/reflectance enhancer 1010, afourth multiplier 123 for multiplying the output of the third multiplier122 and the output of the third corrector 121, and a fourth corrector125 for formatting the output of the fourth multiplier 123 so as toproduce an enhanced output image.

The fourth corrector 125 corrects a brightness value range of the outputof fourth multiplier 123 suitable for the brightness value range of anoutput device.

With regard to a black and white image, the input image f(x,y) can beformatted as the product of the global illumination l_(G)(x,y), thelocal illumination l_(L)(x,y), and the reflectance r(x,y) as equation22.

f(x,y)=l _(G)(x,y)·l _(G)(x,y)·r(x, y)   (22)

Assuming that the global illumination is distributed in the lowestfrequency band, and the reflectance is distributed in the highestfrequency band, the image enhancement scheme, to the generalized imageenhancement of equation 20 obtained from the conventional imageformation model, can be expressed as equation 23:

{circumflex over (f)}(x,y)=g _(1G)({circumflex over (l)} _(G)(x,y))·g_(1L)(l _(L)(x,y))·g ₂({circumflex over (r)}(x,y))   (23)

where {circumflex over (l)}_(G)(x,y) is the estimated globalillumination, {circumflex over (l)}_(L)(x,y) is the estimated localillumination, and {circumflex over (r)}(x,y) is the estimatedreflectance. g_(1G)() and g_(1L)() are functions for decreasing thedynamic ranges of the estimated global illumination {circumflex over(l)}_(G)(x,y) and the estimated local illumination {circumflex over(l)}_(L)(x,y), and g₂() is a function for increasing the contrast ofthe estimated reflectance {circumflex over (r)}(x,y). Typically,g_(1G)(), g_(1L)(), and g₂() are power functions. However, the powerfunctions can be replaced by log functions. {circumflex over (f)}(x,y)is an output image and is processed to adjust its brightness range tothat of the output device, using the fourth corrector 125 represented byg₃() as in equation 21, such that a final output image {tilde over(f)}(x,y) is obtained.

Assuming that a bias is added to the global illumination l_(G)(x,y) andthe local illumination l_(L)(x,y) in equation 22, the globalillumination l_(G)(x,y) and the local illumination l_(L)(x,y) areestimated from the input image f(x,y) by the first and second envelopedetectors 111 and 116 as equations (24) and (25). The reflectance r(x,y)is estimated as equation (26). In equation (24):

{circumflex over (l)} _(G)(x,y)=Env _(G) [f(x,y)]  (24)

{circumflex over (l)}_(G)(x,y) is the estimated global illumination,Env_(G)[] represents the first envelope detector 111 used forestimating the global illumination. From equation 24, it is noted thatthe image obtained by removing the global illumination I_(G)(x,y) fromthe input image f(x,y) is equivalent to the image obtained byAM-modulating the local illumination l_(L)(x,y) using the reflectancer(x,y). The local illumination l_(L)(x,y) is estimated by applying theenvelope detector 116 to an image obtained by dividing the input imagef(x,y) with the estimated global illumination {circumflex over(l)}_(G)(x,y) as in equation (25):

$\begin{matrix}{{{\hat{l}}_{L}\left( {x,y} \right)} = {{Env}_{L}\left\lbrack \frac{f\left( {x,y} \right)}{{\hat{l}}_{G}\left( {x,y} \right)} \right\rbrack}} & (25)\end{matrix}$

where {circumflex over (l)}_(L)(x,y) is the estimated localillumination, Env_(L)[] represents the second envelope detector 116used for estimating the local illumination. The reflectance r(x,y) canbe estimated by dividing the input image f(x,y) with the estimatedglobal illumination {circumflex over (l)}_(G)(x,y) and the estimatedlocal illumination {circumflex over (l)}_(L)(x,y) as in equation (26).

$\begin{matrix}{{\hat{r}\left( {x,y} \right)} = \frac{f\left( {x,y} \right)}{{{\hat{l}}_{G}\left( {x,y} \right)} \cdot {{\hat{l}}_{L}\left( {x,y} \right)}}} & (26)\end{matrix}$

In FIG. 10, the functions g_(1G)() and g_(1L)() decrease the dynamicranges of the estimated global illumination {circumflex over(l)}_(G)(x,y) and the estimated local illumination {circumflex over(l)}_(L)(x,y) as the first and second correctors 119 and 117,respectively. g₂() of the third corrector 121 increases the contrast ofthe estimated reflectance {circumflex over (r)}(x,y). The outputs of thefirst and second correctors 119 and 117 are multiplied by the thirdmultiplier 122, and the output of the third multiplier and the output ofthe third corrector 121 are multiplied such that the output image{circumflex over (f)}(x,y) is produced. g₃() is a function used in thefourth corrector 125 for adjusting the brightness range of the outputimage {circumflex over (f)}(x,y) to that of the output device. Thefourth corrector 125 uses a histogram modeling. However, the histogrammodeling can be replaced by a histogram stretching.

FIGS. 11A and 11B are flowcharts illustrating an image enhancementmethod according to an exemplary embodiment of the present invention. Inthis exemplary embodiment, the image enhancement method is proposed inrelation with the black and white image enhancement system of FIG. 10.How to estimate the global/local illuminations and the reflectance forimproving the quality of the black and white image is describedhereinafter in more detail.

Estimation of Global Illumination

As an envelope detector 111 of equation 28 for estimating the globalillumination, a linear low-pass filtering having wide support region canbe used. The estimation of the global illumination on the input imagef(x,y) using the linear LPF is expressed by equation (27):

$\begin{matrix}{\begin{matrix}{{{\hat{l}}_{G}\left( {x,y} \right)} = {{LPF}_{G}\left\lbrack {f\left( {x,y} \right)} \right\rbrack}} \\{= {\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{f\left( {{x - m},{y - n}} \right)}}}}\end{matrix}\quad} & (27)\end{matrix}$

where h(m,n) is the linear LPF, and W₂ is a 2-dimensional filter window.

In order to perform fast linear low-pass filtering, a Gaussian Pyramidfiltering, which repeats filtering with increasing of the support regionof the filter, is used in this exemplary embodiment.

The support region is increased twice by widening a gap between filtertaps twice without changing the filter coefficients. Whenever the numberof filtering increases, the filtering is performed with wider supportregion. For faster filtering, a 1-dimensional linear LPF is horizontallyapplied to an image and then vertically applied to the result image.Such filtering can be expressed by equation 28:

$\begin{matrix}{{{f^{k}\left( {x,y} \right)} = {\sum\limits_{m \in W_{1}}{\sum\limits_{n \in W_{1}}{{h(m)}{h(n)}{f^{k - 1}\left( {{x - {2^{k - 1}m}},{y - {2^{k - 1}n}}} \right)}}}}},{k = 1},2,\ldots \mspace{11mu},K_{G}} & (28)\end{matrix}$

where f^(k)(x,y) is an image input to the envelope detector 111 and W₁is a 1-dimentional filter window.

If an image is input to the envelope detector 111 (S1142), the inputimage is down sampled (S1143). If the input image is the first one forestimating the global illumination, k is initialized to be set to 0(S1146). At step S1146, K_(G) is the repetition number of filteringoperations for estimating the global illumination. Here, f⁰(x,y)=f(x,y)and f^(K) ^(G) (x,y)={circumflex over (l)}_(G)(x,y).

In this exemplary embodiment, the repetition number of filtering K_(G)of the input image having a size of M×N is decided by equation (29)based on simulation results which have shown that the best performanceis obtained when the maximum support region of the filter is ¼ of the1-dimensional size of the input image. In equation (29):

$\begin{matrix}{K_{G} = \left\{ \begin{matrix}{{E - 1},\mspace{14mu} {E \geq 3}} \\{1,\mspace{14mu} {otherwise}}\end{matrix} \right.} & (29)\end{matrix}$

E is a constant determined by equation (30) in consideration of thetwice increase of the support region of the filter whenever increasingthe number of filtering.

E=log₂ max(M,N)   (30)

The step S1146 is a process for initializing a variable k indicating anumber of filtering iterations for estimating the global illumination.The step S1148 is a process for determining whether the number offiltering iterations reaches a repetition number K_(G) of filteringoperations. If a current number of filtering iterations is not equal tothe repetition number K_(G), low-pass filtering is performed in stepS1152 and the number of filtering increases by 1 (k=k+1) in step S1153.After the low-pass filtering is performed, the increased number offiltering is compared with the repetition number K_(G) in step S1148. Ifk is equal to K_(G) at step S1148, the down sampled image is up sampledso as to be restored its original size in step S1149, and the estimatedglobal illumination is stored in step S1154. FIGS. 12A and 12B areexemplary images before and after applying a linear low passing filterof an image enhancement system according to an exemplary embodiment ofthe present invention. The input image (FIG. 12A) having a size of2000×1312 is filtered by a linear LPF having filter coefficients ofh(m)=h(n)=[0.25,0.5,0.25],m,n=−1,0,1 FIG. 12B illustrates the globalillumination estimated with the repetition number 9 (K_(G)=9). As shownin FIG. 12B, the global illumination varies gradually over the entirearea including edges.

Estimation of Local Illumination and Reflectance

After estimating the global illumination, the local illumination isestimated by removing the estimated global illumination from the inputimage. In order to remove the estimated global illumination, it isinverted by the first inverter 113 and then is multiplied with the inputimage at the first multiplier 115 in step S1156. Accordingly, anintermediate image is obtained by removing the estimated globalillumination from the input image. The intermediate image is downsampled in step S1155. The estimation of local illumination is performedthrough steps S1158 to S1166. In step S1158, k is initialized to be setto 0. k is a counter for indicating the number of filtering iterationsfor estimating the local illumination. The linear low-pass filtering isrepeated to K_(L) in step S1160. If the current number of filteriterations is not equal to the repetition number K_(L), a justnoticeable difference (JND)-based low-pass filtering is performed instep S1162 and the number of filtering increases by 1 (k=k+1) in stepS1164. The JND-based nonlinear low-pass filtering is used for estimatingthe local illumination in a minimum local brightness difference evenwith the irregular image variation. The minimum local brightnessdifference of the local variation is obtained on the basis of Weber'slaw as equation (31):

JND(I)=μ+σ·I   (31)

where I is a uniform brightness value of image, JND(I) is the JND valueto I, and μ and σ are constants. Using JND value, it is possible toobtain the minimum brightness difference JND(f(x,y)) of a brightnessf(x,y) at a specific image pixel (x,y) which the HVS can perceive. Thatis the reason why the minimum brightness difference is used is forrestraining the halo effect. For example, a pixel of which brightnessdifference with that of the center pixel of the filter window is greaterthan JND is excluded in the filtering, and a pixel of which brightnessdifference is less than JND is filtered with a linear LPF, whichperforms filtering by adjusting its filter coefficient according to thebrightness difference level. In this exemplary embodiment, in order toexpress the JND-based nonlinear LPF, an image obtained by removing theestimated global illumination {circumflex over (l)}_(G)(x,y) from theinput image f(x,y) is expressed by f₁(x,y), for simplifying theexplanation, as following equation (32).

$\begin{matrix}{{f_{1}\left( {x,y} \right)} = \frac{f\left( {x,y} \right)}{{\hat{l}}_{G}\left( {x,y} \right)}} & (32)\end{matrix}$

A ratio of the difference if |f₁(x,y)−f₁(x−m,y−n)| between thebrightness of a center pixel (x,y) and the brightness of a pixel distantaway as much as (m,n) from the center pixel, to the JND valueJND(f₁(x,y)) of the center pixel (x,y) can be expressed by J(x,y;m,n) asequation 33.

$\begin{matrix}{{J\left( {x,{y;m},n} \right)} - \frac{{{f_{1}\left( {x,y} \right)} - {f_{1}\left( {{x - m},{y - n}} \right)}}}{{JND}\left( {f_{1}\left( {x,y} \right)} \right)}} & (33)\end{matrix}$

A weighting function λ(x,y;m,n) for adjusting the coefficients of thefilter according to the ratio J(x,y;m,n) of equation (33) can be definedas following equation:

$\begin{matrix}{{\lambda \left( {x,{y;m},n} \right)} = \left\{ \begin{matrix}{0,} & {{{if}\mspace{14mu} {J\left( {x,{y;m},n} \right)}} > 1} \\{{\exp \left\lbrack {- \frac{{J\left( {x,{y;m},n} \right)} - {Th}}{1 - {Th}}} \right\rbrack},} & {{{if}\mspace{14mu} {Th}} \leq {J\left( {x,{y;m},n} \right)} \leq 1} \\{1,} & {{{if}\mspace{14mu} {J\left( {x,{y;m},n} \right)}} < {Th}}\end{matrix} \right.} & (34)\end{matrix}$

where Th is a threshold. The estimation of the local illumination usingthe linear LPF adopting the weighting function λ(x,y;m,n) of equation(34) is expressed as equations (35) and (36).

$\begin{matrix}\begin{matrix}{{{\hat{l}}_{L}\left( {x,y} \right)} = {{LPF}_{L}\left\lbrack {f_{1}\left( {x,y} \right)} \right\rbrack}} \\{= {\frac{1}{\Lambda \left( {x,y} \right)}{\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}{f_{1}\left( {{x - m},{y - n}} \right)}}}}}\end{matrix} & (35) \\{{\Lambda \left( {x,y} \right)} = {\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}}}} & (36)\end{matrix}$

Through equations (31) to (36), in the case that J(x,y;m,n) is greaterthan 1, the pixel is regarded as a pixel of which brightness differencewith the center pixel can be detected, whereby the pixel is not used forfiltering the center pixel (x,y)

In a case that J(x,y;m,n) is less than Th, the pixel is regarded as apixel of which brightness is little different from that of the centerpixel, whereby the pixel is used for filtering the center pixel (x,y)with the original filter coefficient. If Th≦J(x,y;m,n)≦1, the pixel isused for filtering the center pixel with a filter coefficient of whichvalue is decreased from the original filter coefficient, as J(x,y;m,n)closes to 1.

Accordingly, a pixel of which brightness difference with the centerpixel is greater than or equal to JND multiplied by Th uses a nonlinearLPF adjusting the filter coefficient in this embodiment. For performinga nonlinear low-pass filtering, a 1 dimensional filter is applied inhorizontal and vertical directions in alternate fashion as in equations(37) and (38):

$\begin{matrix}{{{f_{1}^{K}\left( {x,y} \right)} = {\frac{1}{{\Lambda (x)}{\Lambda (y)}}{\sum\limits_{m \in W_{1}}{\sum\limits_{n \in W_{1}}{{h(m)}{h(n)}{\lambda \left( {x;m} \right)}{\lambda \left( {y;n} \right)}{f_{1}^{k - 1}\left( {{x - {2^{k - 1}m}},{y - 2^{k - 1}}} \right)}}}}}},} & (37) \\{{\Lambda (x)} = {\sum\limits_{m \in W_{1}}{{h(m)}{\lambda \left( {x;m} \right)}}}} & (38)\end{matrix}$

where K_(L) is the repetition number of filtering operations, λ(x;m) isa 1-dimensional signal of equation 34, Λ(y) is defined by equation (38).At this time f₁ ⁰(x,y)=f₁(x,y) and f₁ ^(K) ^(L) (x,y)=l_(L)(x,y).

After increasing the number of filtering operations, the increasednumber of filtering is compared with the repetition number K_(L) at stepS1160.

If k is equal to K_(L) at step S1160, the filtering operation iscompleted such that the down sampled image is up sampled so as to berestored its original size in step S1159, and the estimated localillumination is stored in step S1166.

In the case that the envelope detector 116 is implemented with JND-basednonlinear LPF, it shows best performance when the maximum support regionof the filter is 1/64 of the 1-dimensional size of the intermediateimage. On the basis of the simulation results, the repetition numberK_(L) of filtering operations to the input image of M×N size isdetermined as equation 39:

$\begin{matrix}{K_{L} = \left\{ \begin{matrix}{{E - 5},} & {E \geq 7} \\{1,} & {otherwise}\end{matrix} \right.} & (39)\end{matrix}$

where E is a constant determined by equation (30). The halo effect in anoutput image obtained from an input image having strong edges can berestrained by the nonlinear low-pass filtering with the adjustment ofthe coefficients of the filter using JND.

If there is little possibility of the occurrence of halo effect due tono storing edge of the input image so that the filtering is implementedwithout JND, the weighting function is set to 1. FIG. 13A illustrates alocal illumination estimated from the input image of FIG. 12A with therepetition number of 5 (K_(L)=5). Here, the input image is filtered by anonlinear LPF of which coefficients areh(m)=h(n)=[0.25,0.5,0.25],m,n=−1,0,1. FIG. 13B illustrates thereflectance estimated from the input image of FIG. 12 a. The constantfor JND of equation (31) is set to μ=4 and σ=0.12301 as disclosed by S.A. Rajala, M. R. Civanlar, and W. M. Lee in “A Second Generation ImageCoding Technique Using Human Visual System Based Segmentation,” in Proc.IEEE Int. Conf. Acoustics, Speech, Signal Processing, vol. 12, Dallas,Tex., April 1987, pp. 1362-1365,” and the value of the threshold Th isexperimentally set to 0.8.

As illustrated in local illumination of FIG. 13A, an area such as edgeshowing abrupt variation maintains the difference of brightness value,while an area such as sky and roof shows little variation in brightnessvalue. Finally, the reflectance r(x,y) in equation 26 is estimated atstep S1168 and then stored at step S1170. The step S1172 is a processfor determining whether the estimation of the global and localilluminations and reflectance is completed, and the step S1174 is aprocess for performing gamma corrections on the estimated global andlocal illuminations and reflectance. The gamma correction are decreasingthe dynamic range of the estimated global and local illuminations andincreasing the contrast of the reflectance. Step S1176 is a process forprocessing the histogram modeling, and step S1180 is a process forgenerating an output image.

Color Image Enhancement Method

In an exemplary embodiment, it is assumed that the illumination iswhite-light. Under the assumption, the illuminations of R, G, and B havethe same value, so the illumination l_(i)(x,y),iε{R,G,B} of equation 18can be expressed by Accordingly, when the illumination is the whitelight, equation (15) can be replaced with equation (40).

f _(i)(x,y)=l(x,y)·r _(i)(x,y),iε{R,G,B}  (40)

If converting the RGB color image represented by equation (40) into HSVcolor image using a RGB/HSV color conversion relationship as proposed byR. W. G. Hunt in “Measuring Color,” New York, Halsted Press, 1989, therespective component images f_(i)(x,y),iε{R,G,B} can be expressed byequations (41), (42), and (43).

f _(H)(x,y)=r _(H)(x,y)   (41)

f _(S)(x,y)=r _(S)(x,y)   (42)

f _(V)(x,y)=l(x,y)·r _(V)(x,y)   (43)

As in equations (41), (42), and (43), it is known that the H and Scomponent images are not related with the illumination, and only the Vcomponent image is affected by the illumination.

Accordingly, by maintaining the H and S component images of the inputcolor image and improving only the V component image, it is possible toobtain the output color image in which the affection of the illuminationis compensated. If the image enhancement is performed in this manner, itis possible to avoid the gray-world violation in the output color imagebecause the hue and saturation are not changed.

FIG. 14 is a block diagram illustrating a color image enhancement systemaccording to an exemplary embodiment of the present invention.

Referring to FIG. 14, an image enhancement system includes an RGB/HSVconverter 1400 for converting RGB of an input image into HSV, a globalillumination estimator 1404 for estimating the global illumination fromthe V component, a local illumination estimator 1406 for estimating thelocal illumination from an output of the global illumination estimator1404 and the V component output from the RGB/HSV converter 1400, areflectance estimator 1408 for estimating a reflectance from the Vcomponent image output from the RGB/HSV converter 1400, an output of theglobal illumination estimator 1404, and the output of the localillumination estimator 1406, a global/local illumination and reflectanceenhancer 1410 performing a gamma correction for decreasing dynamicranges of the global and local illuminations estimated by the global andlocal illumination estimators 1404 and 1406, and performing gammacorrection for increasing a contrast of the reflectance estimated by thereflectance estimator 1408, a V component image enhancer 1416 formultiplying the outputs from the global/local illumination andreflectance enhancer 1410 such that the brightness range of the Vcomponent image is adjust to that of an output device and a HSV/RGBconverter 1402 for converting the H and V component images output fromthe RGB/HSV converter 1400 and the V component image output from the Vcomponent image enhancer 1416 into RGB component images.

In an exemplary embodiment, the input RGB color image is converted intoan HSV color image and then a V component image is enhanced on the basisof the advanced image formation model under the assumption ofwhite-light illumination environment. The global illumination isestimated using a linear LPF having a wide support region from the Vcomponent image at the global illumination estimator 1404, and the localillumination is estimated using a JND-based nonlinear LPF having anarrow support region from an image obtained by removing the globalillumination from the V component image at the local illuminationestimator 1406. The reflectance is estimated by dividing the V componentimage with the estimated global and local illuminations at thereflectance estimator 1408. The estimated components are gamma correctedat the global/local illumination and reflectance enhancer 1410 and thenthe enhanced components are processed at the V component image enhancer1416 such that an enhanced V component image is produced. The gammacorrection is applied in decrementing strengths, i.e. globalillumination>local illumination>reflectance. In order for the brightnessrange of the output V component image to be adjusted to that of anoutput device, the histogram modeling is applied such that the finaloutput V component image is obtained.

Finally, using the final output V component image and the input H and Scomponent images, the HSV/RGB converter 1402 produces an output RGBcolor image.

FIG. 15 is a block diagram illustrating a configuration of an exemplaryimage enhancement system of FIG. 14. FIGS. 16A and 16B are flowchartsfor illustrating an operation of the image enhancement system of FIG.14.

An image enhancement operation is described in association with theestimation of the global and local illumination and reflectance withreference to FIGS. 14 to 16.

In FIG. 16A, it is assumed that the illumination is white-light in thecolor space as an initial condition in step S440. If an R, G, and Bcolor image is input in step S442, the RGB/HSV converter 1400 convertsthe RGB color image into an H, S, and V color image in step S444.

From the input V component image f_(V)(x,y) as an output of the RGB/HSVconverter 1400, an envelope detector 411 of the global illuminationestimator 1404 estimates the global illumination l_(VG)(x,y)representing a global brightness variation. The estimated globalillumination {circumflex over (l)}_(VG)(x,y) is inverted by an inverter413, and the output of the inverter 413 is multiplied with the input Vcomponent image f_(V)(x,y) by a multiplier 415 such that the globalillumination is estimated. The local illumination l_(VG)(x,y)representing the local brightness variation is estimated from an imageobtained by removing the estimated global illumination from the input Vcomponent image f_(V)(x,y) at an envelope detector 416. A reflectancer_(V)(x,y) is estimated by multiplying a value obtained by inverting, atan inverter 418, the estimated local illumination {circumflex over(l)}_(VL)(x,y) with the output of the multiplier 415, at anothermultiplier 420. The output of the envelope detectors 411 and 416processed by correctors 419 and 417 such that their dynamic ranges aredecreased. The outputs of the correctors 419 and 417 are multiplied witheach other at a multiplier 422. The estimated reflectance output fromthe multiplier 420 is processed by a corrector 421 such that itscontrast is increased. The output of the corrector 421 is multipliedwith the output of the multiplier 422 so as to be output as a formattedimage. The brightness range of the output of the multiplier 423 isadjusted to that of an output device by a histogram modeling unit 425such that a final output V component image is output. In the imageformation model of this exemplary embodiment, the input V componentimage f_(V)(x,y) is expressed as a multiplication of the globalillumination l_(VG)(x,y) the local illumination l_(VL)(x,y), and thereflectance r_(V)(x,y) as equation 44.

f _(V)(x,y)=l _(VG)(x,y)·l _(VL)(x,y)·r _(V)(x,y)   (44)

Assuming that the global illumination estimated by the globalillumination estimator 1404 is distributed in the lowest frequency band,and the reflectance estimated by the reflectance estimator 1408 isdistributed in the highest frequency band, the image enhancement ofequation (15) based on the advanced image formation model of equation 22can be expressed by equation (45):

{circumflex over (f)} _(V)(x,y)=g _(1G)({circumflex over (l)}_(VG)(x,y))·g _(1L)({circumflex over (l)} _(VL)(x,y))·g ₂({circumflexover (r)} _(V)(x,y))   (45)

where {circumflex over (l)}_(VG)(x,y) is the estimated globalillumination of the V component image, {circumflex over (l)}_(VL)(x,y)is the estimated local illumination of the estimated V component image,and {circumflex over (r)}_(V)(x,y) is the estimated reflectance of the Vcomponent image. g_(1G)() and g_(1L)() are functions for decreasingthe dynamic ranges of the estimated global and local illuminations{circumflex over (l)}_(VG)(x,t) and {circumflex over (l)}_(VL)(x,y)estimated by the correctors 419 and 417, respectively, and g₂() is afunction for increasing the contrast of the estimated reflectance{circumflex over (r)}_(V)(x,y) estimated by the gamma corrector 421. Thefunctions g_(1G)(), g_(1L)(), and g₂() use power functions for gammacorrections and they can be replaced by log functions. The brightnessrange of the output V component image {circumflex over (f)}_(V)(x,y) isadjusted to that of the output device using the histogram modeling unit425 of g₃() as equation (21) such that the enhanced V component image{tilde over (f)}_(V)(x,y) is obtained.

Assuming that the global illumination l_(VG)(x,y) and local illuminationl_(VL)(x,y) are added by respective biases, the input V component imagef_(V)(x,y) can be regarded that the global illumination l_(VG)(x,y) isAM modulated by the local illumination l_(VL)(x,y) and the reflectancer_(V)(x,y). In this case, the global illumination can be estimated byapplying an envelope detector to the input V component image f_(V)(x,y )as equation 46:

{circumflex over (l)} _(VG)(x,y)=Env _(G) [f _(V)(x,y)]  (46)

where {circumflex over (l)}_(VG)(x,y) is the estimated globalillumination of the V component image, Env_(G)[] is the envelopedetector 411 used for estimating the global illumination. On the basisof equation (26), the image obtained by removing the global illuminationl_(VG)(x,y) from the input V component image f_(V)(x,y) can be regardedthat the local illumination l_(VL)(x,y) is AM modulated by thereflectance r_(V)(x,y). The local illumination l_(VG)(x,y) is estimatedby applying the envelope detector 416 to the image obtained by removingthe estimated global illumination {circumflex over (l)}_(VG)(x,y) fromthe input V component image f_(V)(x,y) as equation 47:

$\begin{matrix}{{{\hat{l}}_{VL}\left( {x,y} \right)} = {{Env}_{L}\left\lbrack \frac{f_{V}\left( {x,y} \right)}{{\hat{l}}_{VG}\left( {x,y} \right)} \right\rbrack}} & (47)\end{matrix}$

where {circumflex over (l)}_(VL)(x,y) is the estimated localillumination, and Env_(L)[] is the envelope detector 416 used forestimating the local illumination. The reflectance r(x,y) is estimatedby dividing the input V component image f_(V)(x,y) by a multiplicationof the estimated global illumination {circumflex over (l)}_(VG)(x,y) andthe estimated local illumination {circumflex over (l)}_(VL)(x,y) asequation (48).

$\begin{matrix}{{{\hat{r}}_{V}\left( {x,y} \right)} = \frac{f_{V}\left( {x,y} \right)}{{{\hat{l}}_{VG}\left( {x,y} \right)} \cdot {{\hat{l}}_{VL}\left( {x,y} \right)}}} & (48)\end{matrix}$

The estimated global and local illuminations {circumflex over(l)}_(VG)(x,y) and {circumflex over (l)}_(VL)(x,y) estimated from theinput V component image f_(V)(x,y) are gamma corrected by the respectivecorrectors 419 and 417 with functions g_(1G)() and g_(1L)() such thattheir dynamic ranges are decreased. The corrector 421 increases thecontrast of the estimated reflectance {circumflex over (r)}_(V)(x,y)with the function g₂(). The outputs of the correctors 419 and 417 aremultiplied by the multiplier 422, and the output of the multiplier 422and the output of the corrector 421 are multiplied by the multiplier 423so as to be formatted as the output V component image {circumflex over(f)}_(V)(x,y). The function g₃() of corrector 425 adjusts the outputimage {circumflex over (f)}_(V)(x,y) to that of an output device.

Estimation of Global Illumination of V Component Image

The f_(V)(x,y), which is an output of the RGB/HSV converter 1400, isinput to the global illumination estimator 1404. The global illuminationestimator 1404 estimates the global illumination l_(VG)(x,y) of the Vcomponent image by envelope detection of the AM modulated V componentimage using the envelop filter 411. The envelope detector 411 ofequation (46) used for the estimation of the global illumination can beimplemented with a linear LPF having wide support region. The estimationof the global illumination l_(VG)(x,y) can be expressed by equation(49).

$\begin{matrix}\begin{matrix}{{{\hat{l}}_{VG}\left( {x,y} \right)} = {{LPF}_{G}\left\lbrack {f_{V}\left( {x,y} \right)} \right\rbrack}} \\{= {\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{f_{V}\left( {{x - m},{y - n}} \right)}}}}\end{matrix} & (49)\end{matrix}$

where h(m,n) is a linear LPF, W₂ is a 2-dimensional filter window.

In an exemplary embodiment, for fast linear low-pass filtering, aGaussian pyramid filtering is performed repeatedly with increasing ofthe support region of the filter. Initially, the filtering is performedwith narrow support region to the input image, and then filtering isrepeated to the image after setting the gaps between the filter tapstwice as wide without changing the filter coefficients. For fastfiltering, a 1-dimensional linear LPF is horizontally applied to animage and then vertically applied to the resulting image. Such filteringoperation can be expressed by equation (50):

$\begin{matrix}{{{f_{V}^{k}\left( {x,y} \right)} = {\sum\limits_{m \in W_{1}}{\sum\limits_{n \in W_{1}}{{h(m)}{h(n)}{f_{V}^{k - 1}\left( {{x - {2^{k - 1}m}},{y - {2^{k - 1}n}}} \right)}}}}},{k = 1},2,\ldots \mspace{14mu},K_{G}} & (50)\end{matrix}$

where f_(V) ^(k)(x,y) is an input V component image obtained by k^(th)linear low-pass filtering, and h(m) and h(n) are 1-dimensional linearLPF applied in horizontal and vertical directions. W₁ is a 1-dimensionalfilter window, and K_(G) is the repetition number of filteringoperations. Here, f_(V) ⁰(x,y)=f_(V)(x,y) and f_(V) ^(K) ^(G)(x,y)={circumflex over (l)}_(VG)(x,y).

If an image is input to the envelope detector, the input V componentimage is down sampled in step S445.

In an exemplary embodiment, the repetition number K_(G) of filteringoperations is determined by equation (51) on the basis of simulationresults which have shown that the best performance is obtained when themaximum support region of the filter is ¼ of the 1-dimensional size ofthe input image:

$\begin{matrix}{K_{L} = \left\{ \begin{matrix}{{E - 1},} & {E \geq 3} \\{1,} & {otherwise}\end{matrix} \right.} & (51)\end{matrix}$

where E is a constant determined by equation (52) in consideration ofthe twice increase of the support region of the filter wheneverincreasing the number of filtering.

E=log₂ max(M,N)   (52)

In the step S446 of FIG. 16A, a variable k indicating a number offiltering operations for estimating the global illumination isinitialized. The step S448 is a process for determining whether therepetition number of filtering operations reaches a repetition numberK_(G). If the current number of filtering operations is equal to therepetition number K_(G), the image is up sampled in step S453, and theestimated global illumination {circumflex over (l)}_(VG)(x,y) is storedin step S454. Alternatively, if the current number of filteringoperations is not equal to the repetition number K_(G), low-passfiltering is performed in step S450 and the number of filteringincreases by 1 (k=k+1) in step S452. After the low-pass filtering isperformed, the increased number of filtering is compared with therepetition number K_(G) again in step S448.

Estimation of Local Illumination of V Component Image

The local illumination l_(VL)(x,y) of the V component image is estimatedby the local illumination estimator 1406. The estimated globalillumination {circumflex over (l)}_(VG)(x,y) estimated by the globalillumination estimator 1404 is inverted by the inverter 413, and theoutput of the inverter 413 is multiplied with the f_(V)(x,y) output fromthe RGB/HSV converter 1400 by the multiplier 415 such that the estimatedglobal illumination {circumflex over (l)}_(VG)(x,y) is removed from thef_(V)(x,y) in step S456. The output of the multiplier 415 is AMdemodulated by the envelope detector 416 such that the localillumination of the V component image is estimated from envelopedetector. At the step S457 of FIG. 16A, the local illumination isestimated. Remaining is a part obtained by removing the estimated globalillumination from the f_(V)(x,y). At the step S458, a variable kindicating the repetition number of the filtering is initialized. TheJND of the local variation in an image is approximated as equation (53)on the basis of Weber's Law:

JND(I)=μ+σ·I   (53)

where I is a uniform brightness value of image, JND(I) is the JND valueof I, and μ and σ are constants. Using the JND, it is possible to obtainthe minimum brightness difference JND(f_(V)(x,y)) which HVS can perceiveat a pixel (x,y) of the image.

A pixel of which brightness difference with the center pixel is greaterthan JND within the filter window is used is the envelope detectorEnv_(L)[] for estimating the local illumination of equation (47) isexcluded for restraining the halo effect in an exemplary embodiment. Apixel of which brightness difference with the center pixel is less thanJND is filtered by a nonlinear LPF which performs filtering by adjustingthe coefficients of the filter according to the difference level of thebrightness. In an exemplary embodiment, the image obtained by removingthe estimated global illumination {circumflex over (l)}_(VG)(x,y) fromthe input V component image f_(V)(x,y) is expressed as equation (54).

$\begin{matrix}{{f_{1V}\left( {x,y} \right)} = \frac{f_{V}\left( {x,y} \right)}{{\hat{l}}_{VG}\left( {x,y} \right)}} & (54)\end{matrix}$

A ratio of the brightness difference |f_(1V)(x,y)−f_(1V)(x−m,y−n)|between the brightness of a pixel distant as much as (m,n) from thecenter pixel (x,y) in the filter window to the JND valueJND(f_(1V)(x,y)) of the center pixel (x,y) is expressed by J(x,y;m,n) asequation (55).

$\begin{matrix}{{J\left( {x,{y;m},n} \right)} = \frac{{{f_{1V}\left( {x,y} \right)} - {f_{1V}\left( {{x - m},{y - n}} \right)}}}{{JND}\left( {f_{1V}\left( {x,y} \right)} \right)}} & (55)\end{matrix}$

According to the value of J(x,y;m,n), a weighting function λ(x,y;m,n)for adjusting the coefficients of the filter is defined by equation(56):

$\begin{matrix}{{\lambda \left( {x,{y;m},n} \right)} = \left\{ \begin{matrix}{0,} & {{{if}\mspace{14mu} {J\left( {x,{y;m},n} \right)}} > 1} \\{{\exp \left\lbrack {- \frac{{J\left( {x,{y;m},n} \right)} - {Th}}{1 - {Th}}} \right\rbrack},} & {{{if}\mspace{14mu} {Th}} \leq {J\left( {x,{y;m},n} \right)} \leq 1} \\{1,} & {{{if}\mspace{14mu} {J\left( {x,{y;m},n} \right)}} < {Th}}\end{matrix} \right.} & (56)\end{matrix}$

where Th is a threshold. The estimation of the local illumination usingthe nonlinear LPF adopting the weighting function λ(x,y;m,n) of equation(56) is expressed as equations (57) and (58).

$\begin{matrix}\begin{matrix}{{{\hat{l}}_{VL}\left( {x,y} \right)} = {{LPF}_{L}\left\lbrack {f_{1V}\left( {x,y} \right)} \right\rbrack}} \\{= {\frac{1}{\Lambda \left( {x,y} \right)}{\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}{f_{1V}\left( {{x - m},{y - n}} \right)}}}}}\end{matrix} & (57) \\{{\Lambda \left( {x,y} \right)} = {\sum\limits_{m \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}}}} & (58)\end{matrix}$

Through equations (55) to (58), a pixel of which J(x,y;m,n) is greaterthan 1 is regarded that its brightness difference with the center pixelcan be detected. In this case, the pixel is not used for filtering thecenter pixel (x,y). A pixel of which J(x,y;m,n) is less than Th isregarded that there is little brightness difference with the centerpixel, such that the pixel is used for filtering the center pixel (x,y)with the original filter coefficient. In the case of Th≦J(x,y;m,n)≦1,the filtering is performed at the center pixel (x,y) using the filtercoefficient which decreases from the original filter coefficient in sizeas J(x,y;m,n) closes to 1. In order for the nonlinear LPF to performfast filtering, a 1-dimensional filter is applied in horizontal andvertical directions in alternate fashion:

$\begin{matrix}{\begin{matrix}{{f_{1V}^{k}\left( {x,y} \right)} = \frac{1}{{\Lambda (x)}{\Lambda (y)}}} \\{{= {\sum\limits_{m \in W_{1}}{\sum\limits_{n \in W_{1}}{{h(m)}{h(n)}{\lambda \left( {x,m} \right)}{\lambda \left( {y,n} \right)}{f_{W}^{k - 1}\left( {m,{y - {2^{k - 1}n}}} \right)}}}}},}\end{matrix}{k = 1},2,\ldots \mspace{14mu},K_{L}} & (59) \\{{\Lambda (x)} = {\sum\limits_{m \in W_{1}}{{h(m)}{\lambda \left( {x;m} \right)}}}} & (60)\end{matrix}$

where K_(L) is a repetition number of filtering operations (see S460),and λ(x;m) is a 1-dimensional signal in equation (60). Λ(y) can bedefined as the equation (60). Here, f_(1V) ⁰(x,y)=f_(1V)(x,y) and f_(1V)^(K) ^(L) (x,y)={circumflex over (l)}_(VL)(x,y). If the number offiltering operations k is less than K_(L) at step S460, the JND-basedlow-pass filtering is performed in step S462 and the number is increasedby 1 in step S464. If the number of filtering operations is equal toK_(L), the image is up sampled in step S463, and the estimated localillumination is stored in step S466. That is, the JND-based filtering isperformed to the repetition number determined by equation (61) in stepS462, and the local illumination estimated through the repeatedfiltering is stored in step S466.

In the case that the envelope detector 416 is implemented with aJND-based filter, simulation results have shown that the filter givesthe best performance when the maximum support region of the linear LPFis 1/64 of the input image in 1-dimensional size. On the basis of thesimulation result, the repetition number K_(L) to the input image of M×Nsize is determined as equation (61):

$\begin{matrix}{K_{L} = \left\{ \begin{matrix}{{E - 5},} & {E \geq 7} \\{1,} & {otherwise}\end{matrix} \right.} & (61)\end{matrix}$

where E is a constant determined by equation (52). The halo effect in anoutput image obtained from an input image having strong edges can berestrained by the nonlinear low-pass filtering with the adjustment ofthe coefficients of the filter using JND. If there is little possibilityof the occurrence of the halo effect due to no storing edge in the inputimage so that the filtering is implemented without JND, a weightingfunction is set to 1. In this case, equation 59 represents a linearlow-pass filtering. Finally, the reflectance r_(V)(x,y) of the Vcomponent image is estimated using equation 48.

Estimation of Reflectance of V Component Image

The reflectance estimator 1408 includes an inverter 418 for invertingthe output of the multiplier 415 through the envelope detector 416, anda multiplier 420 for multiplying the output of the inverter 418 and theoutput of the multiplier 415. This can be expressed with an equation atstep S468 of FIG. 16B. At step S468, the reflectance is estimated andthe estimated reflectance is stored at step S470.

Enhancement of global/local illuminations and reflectance of V componentimage of a color image

If the global and local illuminations and reflectance of the V componentimage are estimated at step S472, a gamma correction is performed on theestimated global and local illuminations and reflectance. In FIG. 15,the global/local illuminations and reflectance enhancer 1410 includescorrectors 419 and 417 for decreasing the dynamic ranges of the globaland local illuminations estimated by the global and local illuminationestimators 1404 and 1406, and a corrector 421 for increasing thecontrast of the reflectance estimated by the reflectance estimator 1408.The V component image enhancer 1416 includes a multiplier 422 formultiplying the outputs of the correctors 419 and 417 of theglobal/local illumination and reflectance enhancer 1410 with each other,a multiplier 423 for multiplying the output of the multiplier 422 andthe output of the corrector 421, and a corrector 425 for correcting thebrightness range of the output from the multiplier 423 to adjust to thebrightness range of an output device.

After the global and local illuminations and the reflectance areestimated in step S472, the dynamic ranges of the global and localilluminations {circumflex over (l)}_(VG)(x,y) and {circumflex over(l)}_(VL)(x,y) estimated from the input V component image f_(V)(x,y) aredecreased by applying the functions g_(1G)() and g_(1L)() of thecorrectors 419 and 417 in step S474, and the contrast of the estimatedreflectance {circumflex over (r)}_(V)(x,y) is increased by applying thefunction g₂() of the corrector 421. Consequently, the outputs of thecorrectors 419 and 417 are multiplied with each other on the basis ofequation 27 by the multiplier 422, and the output of the multiplier 422and the corrector 421 are multiplied with each other such that anenhanced output V component image {circumflex over (f)}_(V)(x,y) isobtained as equation (62):

{circumflex over (f)} _(V)(x,y)=({circumflex over (l)} _(VG)(x,y))^(α) ¹·({circumflex over (l)} _(VL)(x,y))^(α) ² ·({circumflex over (r)}_(V)(x,y))   (62)

where α₁ and α₂ are constants used in the gamma corrections fordecreasing the dynamic ranges of the estimated global illumination{circumflex over (l)}_(VG)(x,y) and the estimated local illumination{circumflex over (l)}_(VL)(x,y), β is a constant used in the gammacorrection for increasing the contrast of the estimated reflectance{circumflex over (r)}_(V)(x,y). These constants are selected to satisfya condition of 0≦α₁≦α₂≦β≦1 in consideration that a low frequency bandmuch affects the dynamic range of an image and the HVS is more sensitiveto the dark region than the bright region. These constants aredetermined according to a taste of a user such as the contrast,brightness, and naturalness of the output color image. The brightnessrange of the output V component image {circumflex over (f)}_(V)(x,y) isadjusted to that of the brightness range of the output device by g₂()as in equation (21) such that the final output V component image {tildeover (f)}_(V)(x,y) is obtained. At step S476, the histogram modelingmethod (A. K. Jain, Fundamentals of “Digital Image Processing,”Englewood Cliffs, N.J., Prentice-Hall, 1989) is used with the functionof g₃() as equations (63) and (64):

$\begin{matrix}{{{\overset{\sim}{f}}_{V}\left( {x,y} \right)} = \left\lfloor {{\frac{\left( {s - s_{\min}} \right)}{1 - s_{\min}}\left( {L_{2} - 1} \right)} + 0.5} \right\rfloor} & (63) \\{{s = \frac{\overset{q}{\sum\limits_{i}}{h_{v}^{\chi}\left( V_{j} \right)}}{\sum\limits_{j = 0}^{L_{1} - 1}{h_{v}^{\chi}\left( V_{j} \right)}}},{q = {{\hat{f}}_{V}\left( {x,y} \right)}}} & (64)\end{matrix}$

where s_(min) is a minimum value of s, V_(j) is the j^(th) brightnessvalue (j=0,1, . . . ,L_(I−1)) of the output V component image {tildeover (f)}_(V)(x,y), and h_(v)(V_(j)) is the histogram of V_(j). χ is aconstant for determining a histogram shape of the final output Vcomponent image {tilde over (f)}_(V)(x,y), and L₂−1 is a maximum valueof the brightness of the final output V component image {tilde over(f)}_(V)(x,y). If χ=0, there is little variation in the histogram shape.If χ=1, the histogram shape closes to a uniform distribution so as toobtain the same result of the histogram equalization method. In order tocompensate the shortcoming of the histogram equalization whilemaintaining the histogram shape of the output V component image {tildeover (f)}_(V)(x,y) to some extent, χ is as a value close to 0 satisfying0≦χ≦1. The brightness range of the final output V component image {tildeover (f)}_(V)(x,y) is determined by L₂ regardless of the output Vcomponent image {circumflex over (f)}_(V)(x,y), whereby it is requiredto select a value of L₂ which enables adjusting the brightness range ofthe final output V component image {tilde over (f)}_(V)(x,y) to that ofthe output device. In an exemplary embodiment, the histogram modelingmethod represented by g₃() of corrector 425 generates the final outputV component image {tilde over (f)}_(V)(x,y) of which histogram shape ismaintained as the output V component image {circumflex over(f)}_(V)(x,y) to some extent while its brightness range is correctedregardless of the output V component image {circumflex over(f)}_(V)(x,y). The histogram modeling shows a better result than usingg₃() for the gain/offset correction in which constants used for thegain/offset correction are set to different values according to theoutput V component image {circumflex over (f)}_(V)(x,y) for adjustingthe brightness range of the final output V component image {tilde over(f)}_(V)(x,y) to that of the output device. In step 478, the finaloutput V component image obtained through these processes is convertedinto the RGB output color image with the original H and S componentimages by the HSV/RGB converter 1402 of FIG. 14 and the image is outputin step 480. FIGS. 17A and 17B illustrate output color images obtainedby enhancing the input image of FIG. 8A according to an exemplaryembodiment of the present invention.

In order to show the effect of JND-based nonlinear low-pass filtering, afinal output color image obtained when a linear low-pass filtering isused without adopting the JND is shown together. The constants used forthe gamma correction in equation (62) are set to α₁=0.2, α₂=0.4, andβ=0.8 that show the best result in the simulations, and the constantsused for the histogram modeling in equation L₂ and χ in equations (63)and (64) are set to 256 and 0.2.

FIG. 17A illustrates an output image obtained by enhancing the inputimage of FIG. 8A without adopting the JND according to an embodiment ofthe present invention. As shown in FIG. 17A, the output image does notshow a color variation, unlike the output image of the MSRCR, andentirely well enhanced. However, the halo effect is shown close aroundthe tower and trees.

FIG. 17B illustrates a final output color image obtained by enhancingthe input image of FIG. 8A with the JND according to an exemplaryembodiment of the present invention. As shown in FIG. 17B, the finaloutput color image obtained by adopting the JND is well enhanced withoutany halo effect.

FIGS. 18A to 18C are partially enlarged images of the final output colorimages enhanced by the image enhance method according to an exemplaryembodiment of the present invention.

As shown in FIG. 18A, the halo effect is widely occurred around thetower and the branches of the trees are shown partially bright andpartially dark due to the halo effect. In FIG. 18B, which is the finaloutput color image that is obtained without adopting JND, the haloeffect caused by the global illumination is restrained. However, anarrow halo effect occurs close around the tower. In FIG. 18C, however,the final output color image obtained by adopting the JND does not showthe halo effect caused by the local illumination. From the images ofFIGS. 17A, 17B, and 18A to 18C, it is known that an exemplary imageenhancement method of the present invention increases both the globaland local contrasts well on the basis of the advanced image formationmodel. Also, it is known that the JND-based nonlinear LPF can restrainthe halo effect, and the output color image can be obtained withoutcolor variation by improving only the V component image of the inputcolor image.

In order to evaluate the performance of an exemplary image enhancementmethod of the present invention, a simulation image database is builtwith RGB color images of size 2000×1312 downloaded from NationalAeronautics and Space Administration (NASA) homepage and RGB colorimages of size 352×288 of Motion Picture Experts Group 7 (MPEG-7) CommonColor Dataset (CCD). The images are processed with the conventionalhistogram equalization method, MSRCR, and the image enhancement methodof the present invention and compared with each other in quality ofoutput images. For fair performance evaluation, in the histogramequalization method, only the V component image of an HSV color imageconverted from an RGB color image is improved, and then the output RGBimage is obtained using the input H and S component images and theimproved V component image, as in an exemplary image enhancement methodof the present invention. In the MSRCR method, all the constants are setto the same values used in an exemplary image enhancement method of thepresent invention.

FIGS. 19A to 19D illustrate a NASA1 input color image, output colorimages obtained by the histogram equalization method, MSRCR method, andan exemplary image enhancement method of the present invention. In FIG.19A, the input image shows grass and moss, dark and weak edges, and ablue surfing board having deep saturation. In the output color image ofFIG. 19B obtained by the histogram equalization method, the globalcontrast is increased such that the sail and sands, that are bright inthe input image, are much brightened, and the contrasts around the darkand weak edges of the grass and moss are decreased. In the output colorimage of FIG. 19C by the MSRCR method, the local contrasts around theweak edge of the grass and moss are increased, but the colors of someparts of the grass are changed from green to violet due to the colorvariation and the saturation of the surfing board is weakened by agray-world violation. Meanwhile, in the output color image of FIG. 19Dby an exemplary image enhancement method of the present invention, thecontrasts of the bright and dark regions of the input color image areentirely maintained while the local contrasts around the dark and weakedges of the grass and moss are well increased.

FIGS. 20A to 20D illustrate a NASA2 input color image and output colorimages obtained by the histogram equalization method, MSRCR method, andan exemplary image enhancement method of the present invention,respectively. In FIG. 20A, the input image shows bright pillars havingstoring edges and dark trees having weak edges. In the output colorimage of FIG. 20B by the histogram equalization method, the brightpillars are much brightened and the dark trees are much darkened suchthat the global contrast of the image is increased, but the localcontrast is poorly increased since the terrace and trees are still dark.In the output image of FIG. 20C by the MSRCR method, the local contrastaround the white pillars and the trees is increased, but the spacesbetween the pillars are dark yet and the trees between the building ofblue roof and the white pillars are partially darkened. In the outputimage of FIG. 20D by an exemplary image enhancement method of thepresent invention, the trees between the building of blue roof and thewhite pillars are not darkened while the contrast is increased as inFIG. 20C.

FIGS. 21A to 21D illustrate a CCD1 input color image and output colorimages obtained by the histogram equalization method, MSRCR method andan exemplary image enhancement method of the present invention,respectively. In FIG. 21A, the input image is dark in most regions andpartially bright in some regions. In the output color image of FIG. 21Bby the histogram equalization method, the dark region around thesawteeth is much darkened and the partially bright regions are muchbrightened due to the increase of the global contrast. However, thelocal contrast of the region around the sawteeth is decreased. In theoutput color image of FIG. 21C by the MSRCR method, the local contrastof the region around the sawteeth is increased, and the saturation ofred steel beam is weakened. Also, the global contrast is not increasedenough as in the output color image of FIG. 21B. In the output colorimage of FIG. 21D by an exemplary image enhancement method of thepresent invention, the local contrast of the area around the sawteeth isincreased without the saturation change while the brightness differencesbetween the shadowed dark region and the partially bright regions aremaintained.

If a color or a black and white image is input through a camera or ananother image acquisition device, the black and white image is processedby the image enhancement system of FIG. 10 through the flow of FIG. 11,and the color image is processed by the image enhancement system of FIG.14 through the flowchart of FIG. 16, such that an enhanced image isoutput to an output device.

The image processing device may include an input unit for receiving ablack and white image or a color image, a memory unit for storingapplication programs for processing the images input through the inputunit, and an image processing unit for estimating global and localilluminations and reflectance of the input image through the input unitand producing an output image by enhancing the estimated components. Theimage processing device processes the black and white image using thesystem of FIG. 10 through the procedure of FIG. 11 and processes thecolor image using the system of FIG. 14 through the procedure of FIG.16.

The image processing device can be adapted to a camera as well asportable devices and other communication devices equipped with a camera.

In an exemplary image enhancement system and method of the presentinvention, the estimated global and local illuminations and reflectanceare enhanced by gamma-corrections and the histogram modeling is appliedto the output V component image that is obtained by multiplying theenhanced global and local illuminations and reflectance.

As described above, the image enhancement system and method of anexemplary embodiment of the present invention converts an input RGBcolor image into an HSV color image and improves only the V componentimage of the HSV color image. From the V component image, the globalillumination is estimated using a linear LPF having wide support regionand then the local illumination is estimated by applying a JND-basednonlinear LPF having narrow support region to an image obtained byremoving the estimated global illumination from the V component image.The reflectance is estimated by removing the estimated global and localcontrasts, from the V component image.

Although exemplary embodiments of the present invention are described indetail hereinabove, it should be clearly understood that many variationsand/or modifications of the basic inventive concepts herein taught whichmay appear to those skilled in the present art will still fall withinthe spirit and scope of the present invention, as defined in theappended claims and their equivalents.

1. An image enhancement method comprising: estimating an illuminationfrom an input image; estimating a reflectance by removing the estimatedillumination from the input image; correcting a dynamic range of theestimated illumination; correcting a contrast using the estimatedreflectance; and adjusting a brightness range of an output imageobtained with the corrected dynamic range and contrast to a brightnessrange of an output device.
 2. The image enhancement method of claim 1,wherein the estimating of the illumination comprises estimating using anenvelope detection.
 3. The image enhancement method of claim 2, whereinthe using of the envelope detection comprises using equation:{circumflex over (l)}(x,y)=Env[f(x,y)] where {circumflex over (l)}(x,y)is an estimated illumination, and f(x,y) is an input image.
 4. The imageenhancement method of claim 1, wherein the estimating of the reflectancecomprises: inverting the estimated illumination; and multiplying theinverted illumination with the input image.
 5. The image enhancementmethod of claim 1, wherein each of the correcting of the dynamic rangeand the correcting of the contrast are performed using at least one of alog function and a gamma function.
 6. The image enhancement method ofclaim 1, wherein the adjusting of the brightness range comprises usingat least one of a gain/offset correction method, a histogram modelingmethod and a histogram stretching method, which method is selecteddepending on a brightness range of the output device.
 7. An imageenhancement system comprising: an illumination estimator for estimatingan illumination from an input image; a reflectance estimator forestimating a reflectance by removing the estimated illumination from theinput image; a first corrector for correcting a dynamic range of theestimated illumination; a second corrector for correcting a contrast ofthe estimated reflectance; and a third corrector for adjusting abrightness range of an output image obtained with the corrected dynamicrange and contrast to a brightness range of an output device.
 8. Theimage enhancement system of claim 7, wherein the third correctorperforms at least one of a gain/offset correction method, a histogrammodeling method and a histogram stretching method.
 9. The imageenhancement system of claim 7, wherein the illumination estimatorcomprises a log function block, a linear low-pass filter (LPF), and anenvelope detector connected in cascade.
 10. The image enhancement systemof claim 7, wherein the illumination estimator comprises an envelopedetector and estimates the illumination by performing an envelopedetection.
 11. The image enhancement system of claim 7, wherein thereflectance estimator comprises: an inverter for inverting the estimatedillumination; and a multiplier for multiplying the inverted illuminationwith the input image.
 12. An image enhancement method comprising:estimating a global illumination of an input image; estimating a localillumination of a first intermediate image obtained by removing theglobal illumination from the input image; estimating a reflectance ofthe input image by removing the estimated local illumination from thefirst intermediate image; correcting the estimated global illumination,local illumination, and reflectance by gamma corrections with differentgamma factors; and adjusting a brightness range of an output imageobtained with the gamma-corrected global illumination, localillumination, and reflectance, to a brightness range of an outputdevice.
 13. The image enhancement method of claim 12, wherein theestimating of the global illumination comprises estimating through anenvelope detection process.
 14. The image enhancement method of claim13, wherein the estimating of the global illumination comprisesestimating using the following equation: $\begin{matrix}{{{\hat{l}}_{G}\left( {x,y} \right)} = {{LPF}_{G}\left\lbrack {f\left( {x,y} \right)} \right\rbrack}} \\{= {\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{f\left( {{x - m},{y - n}} \right)}}}}\end{matrix}$ where {circumflex over (l)}_(G)(x,y) is the estimatedglobal illumination, f(x,y) is the input image, h(m,n) is a linearlow-pass filter (LPF)_, and W₂ is a 2-dimensional filter window.
 15. Theimage enhancement method of claim 12, wherein the estimating of thelocal illumination comprises estimating through an envelope detectionprocess.
 16. The image enhancement method of claim 15, wherein theestimating of the local illumination comprises estimating using thefollowing equation: $\begin{matrix}{{{\hat{l}}_{L}\left( {x,y} \right)} = {{LPF}_{L}\left\lbrack {f_{1}\left( {x,y} \right)} \right\rbrack}} \\{= {\frac{1}{\Lambda \left( {x,y} \right)}{\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}{f_{1}\left( {x,y} \right)}}}}}\end{matrix}$ where {circumflex over (l)}_(L)(x,y) is the estimatedlocal illumination, f_(i)(x,y) is the intermediate image, h(m,n) is alinear low-pass filter (LPF), λ(x,y;m,n) is a weighting function and W₂is a 2-dimensional filter window.
 17. The image enhancement method ofclaim 12, wherein the estimating of the reflectance comprises estimatingby dividing the input image with the estimated global and localilluminations.
 18. The image enhancement method of claim 12, wherein theestimating of the reflectance comprises estimated using the followingequation:${\hat{r}\left( {x,y} \right)} = \frac{f\left( {x,y} \right)}{{{\hat{l}}_{G}\left( {x,y} \right)} \cdot {{\hat{l}}_{L}\left( {x,y} \right)}}$where {circumflex over (r)}(x,y) is the estimated reflectance,{circumflex over (l)}_(G)(x,y) is the estimated global illumination,{circumflex over (l)}_(L)(x,y) is the estimated local illumination, andf(x,y) is the input image.
 19. The image enhancement method of claim 12,wherein the adjusting of the brightness range comprises: decreasingdynamic ranges of the estimated global and local illuminations; andincreasing a contrast of the estimated reflectance.
 20. The imageenhancement method of claim 19, wherein the decreasing of the dynamicranges and the increasing of the contrast are each performed using atleast one of a power function and a log function.
 21. The imageenhancement method of claim 12, wherein the adjusting of the brightnessrange comprises: obtaining an output V component image {circumflex over(f)}_(V)(x,y) using equation {circumflex over(f)}_(V)(x,y)=g_(1G)({circumflex over (l)}_(G)(x,y))·g_(1L)({circumflexover (l)}_(L)(x,y))·g₂({circumflex over (r)}(x,y)) where g_(1G)() andg_(1L)() are functions for decreasing the estimated global and localilluminations {circumflex over (l)}_(G)(x,y) and {circumflex over(l)}_(L)(x,y); and adjusting the brightness range of {circumflex over(f)}_(V)(x,y) to the brightness range of the output device.
 22. A colorimage enhancement method comprising: converting RGB component images ofan input image f(x,y) into HSV component images; estimating a globalillumination l_(VG)(x,y) from a V component image f_(V)(x,y) among theHSV component images; estimating a local illumination l_(VL)(x,y) usingthe estimated global illumination {circumflex over (l)}_(VG)(x,y) and abrightness component image f_(V)(x,y); estimating a reflectance r(x,y)using the estimated local illumination {circumflex over (l)}_(VG)(x,y)and the brightness component image f_(V)(x,y) enhancing the estimatedglobal and local illuminations and the estimated reflectance bydecreasing dynamic ranges of the estimated global and localilluminations and increasing a contrast of the estimated reflectance;outputting an enhanced V component image by adjusting a brightness rangeof an output component image obtained by multiplying the enhanced globaland local illuminations and the enhanced reflectance to a brightnessrange of an output device; and reproducing an output color image byconverting the enhanced brightness component image along with the H andS component images into an RGB component images.
 23. The color imageenhancement method of claim 22, wherein estimating of the globalillumination comprises estimating through an envelope detection whichcomprises a kind of AM demodulation.
 24. The color image enhancementmethod of claim 23, wherein the envelope detection comprises usingequation:{circumflex over (l)} _(VG)(x,y)=Env _(G) [f _(V)(x,y)] where{circumflex over (l)}_(VG)(x,y) is the estimated global illumination,and f_(V)(x,y) is the V component image.
 25. The color image enhancementmethod of claim 22, wherein the estimating of the local illuminationl_(VL)(x,y) comprises estimating through an envelope detection whichcomprises a kind of AM demodulation.
 26. The color image enhancementmethod of claim 25, wherein the envelope detection comprises usingequation:${{\hat{l}}_{VL}\left( {x,y} \right)} = {{Env}_{L}\left\lbrack \frac{f_{V}\left( {x,y} \right)}{{\hat{l}}_{VG}\left( {x,y} \right)} \right\rbrack}$where {circumflex over (l)}_(VL)(x,y) is the estimated localillumination, {circumflex over (l)}_(VG)(x,y) is the estimated globalillumination, and f_(V)(x,y) is the V component image.
 27. The colorimage enhancement method of claim 22, wherein estimating of thereflectance comprises estimating with equation:${{\hat{r}}_{V}\left( {x,y} \right)} = \frac{f_{V}\left( {x,y} \right)}{{{\hat{l}}_{VG}\left( {x,y} \right)} \cdot {{\hat{l}}_{VL}\left( {x,y} \right)}}$where {circumflex over (l)}_(VL)(x,y) is the estimated localillumination, {circumflex over (l)}_(VG)(x,y) is the estimated globalillumination, and f_(V)(x,y) is the V component image.
 28. The colorimage enhancement method of claim 22, wherein the enhancing of theestimated global and local illuminations and the estimated reflectancecomprises: correcting the global and local illuminations to decreasedynamic ranges; and correcting the reflectance to increase contrast. 29.The color image enhancement method of claim 28, wherein the correctingof the estimated global and local illuminations and the estimatedreflectance comprises using at least one of a power function and a logfunction.
 30. The color image enhancement method of claim 29, whereinthe outputting of the enhanced V component image comprises: obtaining anoutput V component image {circumflex over (f)}_(V)(x,y) by equation:{circumflex over (f)} _(V)(x,y)=({circumflex over (l)} _(VG)(x,y))^(α) ¹·({circumflex over (l)} _(VL)(x,y))^(α) ² ·({circumflex over (r)}_(V)(x,y))^(β) where α₁ and α₂ are constants used in gamma correctionsfor decreasing the dynamic ranges of the estimated global and localilluminations {circumflex over (l)}_(VG)(x,y) and {circumflex over(l)}_(VL)(x,y), β is a constant used in the gamma correction forincreasing the contrast of the estimated reflectance {circumflex over(r)}(x,y); and adjusting the brightness range of {circumflex over (f)}hdV(x,y) to the brightness range of the output device by equations:${{\overset{\sim}{f}}_{V}\left( {x,y} \right)} = {\left\lfloor {{\frac{\left( {s - s_{\min}} \right)}{1 - s_{\min}}\left( {L_{2} - 1} \right)} + 0.5} \right\rfloor \mspace{14mu} {and}}$${S = \frac{\sum\limits_{j = 0}^{q}{h_{v}^{\chi}\left( V_{j} \right)}}{\sum\limits_{j = 0}^{L_{2} - 1}{h_{v}^{\chi}\left( V_{j} \right)}}},{q = {{\hat{f}}_{V}\left( {x,y} \right)}}$where s_(min) is the minimum value of s, V_(j) is a histogram, χ andL₂−1 are constants determining the histogram shape of the {tilde over(f)}_(V)(x,y) and the maximum value of the brightness, respectively. 31.The color image enhancement method of claim 30, comprising setting α₁and α₂ to values that can affect the dynamic range of low frequency bandmore and satisfy a condition of 0≦α₁≦α₂≦β≦1 in consideration of thehuman visual system more sensitive to dark regions than bright regions.32. The color image enhancement method of claim 30, comprising setting χclose to 0 in a range of 0≦χ≦1.
 33. An image enhancement systemcomprising: a global illumination estimator for estimating a globalillumination from a value (V) component image of an input image; a localillumination estimator for estimating a local illumination using the Vcomponent image and the estimated global illumination; a reflectanceestimator for estimating a reflectance using the V component image, theestimated global illumination, and the estimated local illumination; aglobal/local illumination and reflectance enhancer for performing gammacorrections on the estimated global and local illuminations to decreasethe dynamic ranges and performing a gamma correction on the estimatedreflectance to increase contrast of the estimated reflectance; and abrightness corrector for outputting an output V component image obtainedby multiplying the components output from the global/local illuminationand reflectance enhancer and adjusting the brightness range of theoutput V component image to a brightness range of an output device. 34.The image enhancement system of claim 33, wherein the globalillumination estimator comprises an envelope detector.
 35. The imageenhancement system of claim 34, wherein the envelope detector comprisesa linear low-pass filter (LPF) having wide support region.
 36. The imageenhancement system of claim 35, wherein the linear low-pass filter (LPF)comprises a filter represented by equation: $\begin{matrix}{{{\hat{l}}_{VG}\left( {x,y} \right)} = {{LPF}_{G}\left\lbrack {f_{V}\left( {x,y} \right)} \right\rbrack}} \\{= {\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{f_{V}\left( {{x - m},{y - n}} \right)}}}}\end{matrix}$ where m denotes the index of horizontal direction, ndenotes the index of vertical direction, and W₂ is a 2-dimensionalfilter window.
 37. The image enhancement system of claim 36, wherein thelinear low-pass filter comprises a filter represented by followingequation for 1-dimensional fast filtering:${{f_{V}^{k}\left( {x,y} \right)} = {\sum\limits_{m \in W_{1}}{\sum\limits_{n \in W_{1}}{{h(m)}{h(n)}{f_{V}^{k - 1}\left( {{x - {2^{k - 1}m}},{y - {2^{k - 1}n}}} \right)}}}}},{k = 1},2,\ldots \mspace{14mu},K_{G}$where k is the number of filtering, m denotes the index of horizontaldirection, n denotes the index of vertical direction, W₁ denotes a1-dimensional filter window, and K_(G) is a repetition number offiltering
 38. The image enhancement system of claim 37, wherein therepetition number of filtering operations K_(G) is determined byequation: $K_{G} = \left\{ \begin{matrix}{{E - 1},} & {E \geq 3} \\{1,} & {otherwise}\end{matrix} \right.$
 39. The image enhancement system of claim 38,wherein E is determined at each filtering operation by equation:E=log₂ max(M,N) where M and N are the horizontal and vertical sizes ofimage, respectively.
 40. The image enhancement system of claim 37,wherein the linear low-pass filter (LPF) comprises: a just noticeabledifference (JND) recognizer for recognizing a minimum brightnessdifference (JND) which human visual system can perceive at a pixel ofthe image; and a filter coefficient adjuster for excluding a pixel ofwhich brightness difference with that of a center pixel in a filterwindow is greater than the JND and adjusting coefficients of the filterin accordance with the brightness difference level of a pixel of whichbrightness difference with that of the center pixel is less than JND.41. The image enhancement system of claim 33, wherein the localillumination is estimated by applying a nonlinear low-pass filter (LPF)to an intermediate image obtained by removing the estimated globalillumination from the V component image with following equations:$\begin{matrix}{{{\hat{l}}_{VL}\left( {x,y} \right)} = {{LPF}_{L}\left\lbrack {f_{1V}\left( {x,y} \right)} \right\rbrack}} \\{= {\frac{1}{\Lambda \left( {x,y} \right)}{\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}{f_{1V}\left( {x,y} \right)}}}}}\end{matrix}$ and${\Lambda \left( {x,y} \right)} = {\sum\limits_{{({m,n})} \in W_{2}}{{h\left( {m,n} \right)}{\lambda \left( {x,{y;m},n} \right)}}}$where J(x,y;m,n) is a ratio of a difference|f_(1V)(x,y)−f_(1V)(x−m,y−n)| between a brightness of a center pixel(x,y) and a brightness of a pixel distant away as much as (m,n) from thecenter pixel to a JND of the center pixel (x,y), and λ(x,y;m,n) is aweighting function for adjusting the coefficients of the filteraccording to a value of J(x,y;m,n), and W₂ is a 2-dimensional filterwindow.
 42. The image enhancement system of claim 41 wherein thenonlinear low-pass filter (LPF) is a 1-dimensional filter and thebrightness is obtained by following equations:${{f_{1V}^{k}\left( {x,y} \right)} = {\frac{1}{{\Lambda (x)}{\Lambda (y)}}{\sum\limits_{m \in W_{1}}{\sum\limits_{n \in W_{1}}{{h(m)}{h(n)}{\lambda \left( {x;m} \right)}{\lambda \left( {y;n} \right)}{f_{1V}^{k - 1}\left( {{x - {2^{k - 1}m}},{y - {2^{k - 1}n}}} \right)}}}}}},{k = 1},2,\ldots \mspace{14mu},{k_{L}\mspace{14mu} {and}}$${\Lambda (x)} = {\sum\limits_{m \in W_{1}}{{h(m)}{\lambda \left( {x;m} \right)}}}$where k is a number of filtering, m denotes the index of horizontaldirection, n denotes the index of vertical direction, W₁ denotes a1-dimensional filter window, and K_(L) is a repetition number offiltering operations.
 43. The image enhancement system of claim 42,wherein the non-linear filter comprises a JND-based low-pass filter(LPF).
 44. The image enhancement system of claim 43, wherein theJND-based low-pass filter (LPF) operates with equation:${\lambda \left( {x,{y;m},n} \right)} = \left\{ \begin{matrix}{0,} & {{{if}\mspace{14mu} {J\left( {m,n} \right)}} > 1} \\{\exp \left\lbrack {- \frac{{J\left( {x,{y;m},n} \right)} - {Th}}{1 - {Th}}} \right\rbrack} & {{{if}\mspace{14mu} {Th}} \leq {J\left( {m,n} \right)} \leq 1} \\{1,} & {{{if}\mspace{14mu} {J\left( {m,n} \right)}} < {Th}}\end{matrix} \right.$ where Th is a threshold value.
 45. The imageenhancement system of claim 33, wherein the reflectance is estimatedwith equation:${{\hat{r}}_{V}\left( {x,y} \right)} = \frac{f_{V}\left( {x,y} \right)}{{{\hat{l}}_{VG}\left( {x,y} \right)} \cdot {{\hat{l}}_{VL}\left( {x,y} \right)}}$where {circumflex over (l)}_(VG)(x,y) and {circumflex over(l)}_(VL)(x,y) are the estimated global and local illuminations, andf_(V)(x,y) is the V component image.
 46. A color image enhancementsystem comprising: an RGB/HSV converter for converting RGB componentimages of an input image into HSV component images; a globalillumination estimator for estimating a global illumination from a Vcomponent image among the HSV component images; a local illuminationestimator for estimating a local illumination using the V componentimage and the estimated global illumination; a reflectance estimator forestimating a reflectance using the V component image, the estimatedglobal illumination, and the estimated local illumination; aglobal/local illumination and reflectance enhancer for performing gammacorrections on the estimated global and local illuminations to decreasethe dynamic ranges of the global and local illuminations and performinga gamma correction on the estimated reflectance to increase a contrastof the estimated reflectance; a brightness corrector for outputting anoutput image obtained by multiplying the components output from theglobal/local illumination and reflectance enhancer and adjusting abrightness range of the output V component image to a brightness rangeof an output device; and an HSV/RGB converter for reproducing an outputcolor image by converting the enhanced V component image along with theH and S component images into an RGB component images.
 47. A color imageenhancement method comprising: configuring a white-light illuminationcondition; converting red, green, and blue (RGB) component images intohue, saturation, and value (HSV) component images under assumption ofthe white-light illumination condition; estimating global and localilluminations and reflectance from a V component image of the HSV colorimage; correcting the global and local illuminations and reflectance forenhancing the V component image; adjusting a brightness of an outputcomponent image to a brightness range of an output device; andconverting an HSV image obtained by combining the enhanced V componentimage with H and S component images into an output RGB color image. 48.The color image enhancement method of claim 47, wherein the estimatingof the global illumination comprises: setting an initial value k=0 and arepetition number K_(G) of filtering to the V component image; andestimating the global illumination by repeating the filtering operationin K_(G) times.
 49. The color image enhancement method of claim 47,wherein the estimating of the local illumination comprises: removing theestimated global illumination from the V component image; setting aninitial value k=0 and a repetition number K_(L) of filtering to anintermediate image obtained by removing the estimated globalillumination from the V component image; and estimating the localillumination by repeating the filtering operation in K_(L) times. 50.The color image enhancement method of claim 47, wherein the estimatingof the reflectance comprises removing the global and local illuminationsfrom the V component image.
 51. The color image enhancement method ofclaim 47, wherein the correcting of the global and local illuminationsand reflectance comprises using at least one of a power function and alog function.
 52. The color image enhancement method of claim 47,wherein the correcting of the brightness range of the output V componentimage comprises using at least one of a histogram modeling method and ahistogram stretching method.
 53. The color image enhancement method ofclaim 47, wherein the V component image (f_(V)(x,y)) among the Hcomponent image ((f_(H)(x,y)), S component image (f_(S)(x,y)), and Vcomponent image (f_(V)(x,y)) obtained from the RGB color image isexpressed by equation:f _(V)(x,y)=l _(VG)(x,y)·l _(VL)(x,y)·r _(V)(x,y) where l_(VL)(x,y) isthe local illumination, l_(VG)(x,y) is the global illumination, andr_(V)(x,y) is the reflectance.
 54. The color image enhancement method ofclaim 53, comprising enhancing the brightness component image f_(V)x,y)using at least one of a gamma function and a log function as equation:{circumflex over (f)} _(V)(x,y)=g _(1G)({circumflex over (l)}VG(x,y))·g_(1L)({circumflex over (l)} _(VL)(x,y)) ·g ₂({circumflex over (r)}_(V)(x,y)) where {circumflex over (l)}_(VL)(x,y) is the estimated localillumination, {circumflex over (l)}_(VG)(x,y) is the estimated globalillumination, {circumflex over (f)}_(V)(x,y) is an output V componentimage, and {circumflex over (r)}_(V)(x,y) is the estimated reflectance.55. An image enhancement system having an image input means, comprising:an image processing unit for estimating global and local illuminationsand reflectances of an image input through the input means and forenhancing the image by correcting the estimated components; a storageunit for storing the input image and an output image enhanced by theimage processing unit; and a display unit for displaying the outputimage.
 56. The image enhancement system of claim 55, wherein the imageprocessing unit comprises: a global illumination estimator forestimating the global illumination of the input image; a localillumination estimator for estimating the local illumination of theinput image; a reflectance estimator for estimating the reflectance ofthe input image; a corrector for performing gamma correction fordecreasing dynamic ranges of the estimated global and localilluminations and increasing a contrast of the estimated reflectance;and an output unit for outputting an image enhanced by the corrector toadjust a brightness range of the enhanced image to a brightness range ofthe display unit.
 57. An image enhancement system having an input unitfor acquiring a color image, comprising: a color image processing unitfor converting red, green, and blue (RGB) component images of a colorimage input through the input unit into hue, saturation, and value (HSV)component images, for estimating a global and local illuminations andreflectance from the V component, for enhancing the image by correctingthe global and local illuminations and reflectance and for producing anoutput image by converting the H, S and the enhanced V components intoRGB component images; a storage unit for storing the input color imageand the enhanced output color image; and a display unit for displayingthe output color image.
 58. The image enhancement system of claim 57,wherein the color image processing unit comprises: an RGB/HSV converterfor converting the RGB component images of the input image into HSVcomponent images; a global illumination estimator for estimating theglobal illumination from a V component image; a local illuminationestimator for estimating the local illumination from the estimatedglobal illumination and the V component image; a reflectance estimatorfor estimating the reflectance from the estimated local illumination,the global illumination, and the V component image; a corrector forperforming gamma corrections on the estimated global and localilluminations to decrease the dynamic ranges of the global and localilluminations and for performing a gamma correction on the estimatedreflectance to increase a contrast of the estimated reflectance; abrightness corrector for outputting an enhanced output value componentimage obtained by multiplying the components output from theglobal/local illuminations and reflectance enhancer and adjustingbrightness range of the output value component image to a brightnessrange of an output device; and an HSV/RGB converter for reproducing anRGB image by converting the enhanced V component image along with the Hand S component images into the RGB component image.